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This module was developed as one of eight dissemination 
pclckages wlaich were being prepared undqr an E*S,£.A. l^itle III 
project. The Mesa Putriic Schools. v;orked in concert with the 
Ameriaan Institutes for. Research (AIR) in this Title III. project. 

It should be no ted , that v/hat is 'being presented ' here* is^ in*" 
formation on Mesa's on-goin'g long-range project in career guidance, 
counseling, pla.cement"and follow.-up; funded not only by Title III 
but also by Vocational BtJucation Part D Exemplary and District 
funds. A key element of this project has been the designing, 
f:j^eld testing and final production of s taff development training ' 
packages, ✓ ' , " ' • 

The -spe^rif ic p^articipation of ^^E . S^. E . A . Title III comprises'. 
■ an', integral part of t^ total process for or ienting counseling 
services tov/ard specifffc student outcomes in an accou^jtability ' ' 
model. Title III is housed in-^'the Ari2ona , Department of. -Education 
under Carolyn Warner, Superintendent of Public ' Instruction . The ' 
Title III staff v/as directed by Fred' J. Sughrxre* and the consult- 
ants assigned to this project were Jev/ell Sisemore, the Assistant 
■Director of Title III, and Jesse' Udall,. Education Program* Spe- . 
cialist. t 



.For additional information on these projects, 
•Write or call; (602) 962-7.331 - 



'Exemplary Project Director 
Byron E. MqKinnon 
'Guidance Director 
Mesa Public Schools -. : * 
Guidance, Services Center 
14 0^ S. Center 
Mesa, Arizona 85202 



Title. Ill Director 
DUane Richins ■ 
Guidance Specialis"t 
Mesa Public Schools 
Guidance Services -Center 
140 . S.- Center 
Mesa, .Arizona 85202 
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OVERVIEW OF MODULE- - 

This "mo<;iule" is" part of a packa9^e v/hich' includes the fo.llov;ing: 

Tap.e-s Iii3i3. . pr es-an tation - • 

' ./ - Flowchart , of the comprehensive approach 

- Modulo goals and olSjectaves 
IT Instructional raaterials - ("tes t of module) 
■-.Progress ch6cl;s ' ' . 

- Group activities : 

- Simulation activity 

- /ipplication procedures 

- Pre-' and Post-Assessment 

- Furthdr references 

Packages are designed scfy tliat you may v/ork at your ovm pace 
hkt nhould. n -e-t— txt-k€!^ t>ver-- 15 hours to '':'coTnp.lete . The only elemen t 
■ of the package ox ternal to the 'module is the tape-slide introduction 

_ _ . • ■ The. „f iQV/char-t -:>pay e- d^ f a^ne s the f^u-r--ma >g r- pha^i2s-~trf--^contpt^ 

hensive approach to guidance, counseling, and placemen t and how they 
•relate to each othe^. The darkened segmen*t indicates where this 
.package. Assessing Desired Outcomes, fits in the overall plan. 
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. "h'^" -yon l,ave oompieted th,. ■ ^ ' ' ' " ' ' 



. . . ^ , onsliip among the 

Write a na'rn • ' ■ • i' 

List and doW(-riN • ' . * — ^ * ' 

the dimcsiISs on i^" - ^ =*">tenoc each „f • . 
apprpachls%^;— *si.ea. oJ?^L"^|J|.. . 

•^-''Plain five n i <- ' • 

•assessment aDm-^''''f*'"'^^^ ^esifed o.,*- •■• ' 
past -,V,;i ^P^°^<^^'»es th^Tf- h "^^^otties 
P'^'sc, and state i "^^e been u^J^ ■ 

^- i-ist. the seven - • ' - 

co^e .asspssmentl?;^^"^°'^°n to..a . / 



2.2-27 j. 



3^5- 3i6' 5' 



■ i 
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37-39 6. Identify', purpos^e and importance of • plan- ■ 
^ ;• vning and coordinating" the desired outcome's 
■ _ ■ ' assessment wit It the" advisory group (s), andr^ : 

■ lis t. the iifincipal- activities necessary , to 
... • ' r' accompl.isV •'this ^ step • . ' ' 

43-^4' "7. List three types of important factors that • 
■ " should 43e-considGred in ^-id.cntif ying the areas ' 

-V oji which' the desired outcomes assessment will 
' . ■ ■ i^Qu§ . ' ■ • ; ■ ■ ' ^- . '. • , 

^48r50 . '8. ■ Define: "TaTy^r'^JW importai|t cohsi^^^^ •' ■ 

' in selecting the^-^amplre / . (b) two-'alterhativrs 

: , . ; . sampling stra'tegiek,"^^' (c) describe the; proc.e-. 
' . * .dure 'mbst -appropriate^for yourVsituation/ an 
/ • - ^(d^^taf e the rationale .for ' this choice. - 

' ' ' ' . ' — ^^^:-.^;/V.., \ 

::57-:61 i^ist: (a) three important consider^ltions in . 

. /■ ' developing ah instrument and procedures for 

: o / ^ desired; oiiicomG^s as ^ ^ 

■ ; in .this ' deve.lopraent> and (c) at x least 10 
^ - examples of items you have gener^ated fo.r. your 
instrument. ' *>. • .^k, .. 

71-79 '^j^' %is t^^feh ree amp arj:3.nJi-^nnsidera4:j^ 
■ ' istering.:the instrument (s) you have develbped.- 

. *85-89 * li- fdj^ntafy '.three ap^ inyoived 
. . summar.izingr and ana;iy^ing-"the-data you^'have.' 

*. V connected. : ' ' ' ' * . ■ ' ■ • 



9 1 - 9.4 ^ -i ? • S umma r i z e ~TTi r^eTHiTpprl^^ i d-excrfctons— if{- 



^"ipplying .the results of your . data analysis ' to 
a definition of the. desired outcomes for your 
■' program^ ...... * . • 

/ ... , ■ ■ * * ' . 

92-94 -.-:-4:3. Identifyr^H-a) -are-^tears^^ targe fe^'aTIdri^ 

.of your report wh.iuh summarize the procedure 
... . and products of your desired outcomes assess-; 
. ment; (b) the/topics of , your report, and. (cj^ ' 
\ how the report content "will be tailored for , 

' * . . eaeh target audience. . ' 



MODULE/OUTLINE • 



Approximate 
Time .* 



Activity 



1 hour 



30 minutes 



7 hours . 



30 minutes 

1 hour 
2*^ hours 



Introducfcion , You take and score the pre-- * * 
assessment instrument and view a.^tape-alid.e 
presentation. Next / ^ou examine the flow-* • 
chart for understanding ' of tho^relationsh-i^y 
of this ^module to the overall aompreh<m6ive,. 
program and sg;?ies o/^r-^if^l^agcs , Finaij^ /.you 
are .presentee^ with the module putline wKuch 
shows - approximate 'timelin^ps. . * % : 

Rationale. You are presented with reasons 

the module' was produ<:ed and the purposes it * 

• hopes to serve. Progress checks and -group 
act^jLvitiec are. provided . to ,ilssist- you- in 
attaining/the skills and understanding out- I 

, lined in th-e Bf^dule. .. ' ' ' ^ 

In-Depth Study^ You exte-nd your -knowledge" ' 
4 -hrougli -siriiQy of t^he- body of. tlie package,, 
grottp Activities ^ and interested afeas^ from ■■ . 
€he. te;^erences' sectiion. . » : 

. * .'-.*• • • ■ . . • • . . 

gimulation .. . This activity gives you a chance 
to practice what you have le'arn^d^^* the 
reading. * • 

Application . ^ In this activity you experience- ■ 
the application of thd skills you have gained.-- 

, ^st^ Assessment . riere Is where you demO^lstrate 
that . you^oV^^^ax^hie-Vi^ dt^j ec t.ive s. . of t hfe 
module. ' " • 



l)EFINITlbN OF TERMS 



^ -Some n^ iBi Wi tt- terms used in this module may beruiew to you-r^v/hile 

o-thei^.may be used in ways* that vary from traditional - usage. . It \ 

is important to'. understand- the definitions of terms as they are 

used iri -this module. \ . 

■ ' • • • ' • ■ • ■ 

Asisessmcnt --Thc empirical c?st.imat ion of a state or condition., 
jappli<id in this 'C£\^ to the* current status and desirec^dut.comes .of 
a guidance, couns'eling";. and placeme:nt program. ' . 

• ' Caf'd' -Sort — An assessment tool wh^ch proyidcs an Jjidiyidual 
with a grbup.of s-tatements on sore -feopic and ^ask.s him to pick'.thosQ 
that' -rteld^e to him and. arrange his' choices in the order of their 
imppr-tance. - • , ' \ . . . ' 

Criticar ylncidcnt Tcchnique --'"Criticat incident's^ are examples 
of. ; obi^ervable beliaviora with a plb^r. intent or purpose; and at. 
least one .observable outcome. The -Crit-ical Incident Technique is 
an assessment, procedure which employs these.' 



' >^ Current Status -^The objectives, activities, resfifearces , . arid 

utcomes of a program as it now exists. ' . • • 

*.■■■■ ' . . ' \ 

Delphi: Method — A questidrmaire technique which organizes and 
-shapes, opinion through a system of feedback. 

. - • ■ • ' * . ' «.'.,. 

, • . Oesiyed Outcomes T"Th.c. p^foducts of a theoretrcally. perfect 
program which "completeiy satisfies alF client needs^ and wants'. 
■ . ^ * ' ■ • . . ■ - s . ' • . - * • " . 

\,>. ,Fol low-Up Study — survey' of individuals now removed from a 
situation, or program (in this case, a s6hool and its guidance, 
counseling*, ^'aiTd placement program) which ^colLects t'heir opinipns 
about that situation or p^rogram. f / ^ ~^ 

Need — This term usually applies to individucils*, . but programs 
may be considered to have needs also. In. either case, a need is 
the gap between the current sta^te and the .ide'al . state. The , Concept 
may. be represented graphically. If point A is where, someone is, . 
and point. B is where he would like to be, the gap between these two 
poinTrs--~4^5-_£J3eneed. ^ • ^ . . 

• ' A ^ — ~ — ^ need B ' • • 

. Current ^^^^^^^^--^ , • Desired . 

State ■ ' ' ^----^M' ^ state 

In terms of guidance, counseling and pliacement programs, "A" " 
.represents the pr^ogram's current status and '*B*' its 'des'ired but-* 
Comes. Again, the need is the difference. — . 

* QuestiQnnai-re --AnV of a variety of paper and pencil instruments 
to, which individuals respond to items by cVlecking ^ne of sev^eral 
choices , . rating' their feelings along a scale> checking yes br*'no, ' 
writing -out their response/ or by some similar method.. ■ * 



1 ' Sofnje CO ifusion may oxist- among the terms, current s tr.atus -assess- 
ncntv desired outcomes assessment , and needs assi^ig^smSntT Tradi-- 
.:ionally> the term needs assessment (a .first step in planning for 

•program development or /improvement) has been used to descjiibe a pro- 

* * • . ' 

cess which identifie's the problems \'hat jbrograjiis Should. try to solve. 

I^ut to provide useful in.f ormati9n for program improvement, this pro- 
.cjess shbuld include two measurements: one of where the program now 
'stands (current status assessment), and, one of where it should be 

ipeally ■ (desired outcomes assessment) . Thus, three .training modiiles 

' .r • ' ' ' . • ' . . . _ ^ . . ■ . V 

'h^ve been written:, one to measure the program's current status, one 
.■to measure a program' s desired outcomes, and one , to detei-mine the 
geip betwe^a^these two. (the program' s, needs) a^nd to d^-ive gc^als and 
objectives for the prog^ram £rom its needs.' The- planner who is de- 

velbping an entirely newXprogram will not, of course, cjoneluct a**' 

' ' - ^ ^ . • ■■ r ' ' ."^ ^ 

current status assessment. He will determine program, needs mainly 

f rom.^desired outcomes assessment dat^. . * 

The. desire d outcomes assessment process for a program - relies 

:Lnly on information about stijdents ' needs.*. Theoretically, students 

could define their* desired outcomes — the goalls tov-^ard which they 

aspire, and their current^status could ba assessed relcited .to each 

of their cSebired outcomes. Then a subtraction process could be con-^ 

ducted to determine students' needs. Usually, it is more fieasible- to 

ask ^students to define Jiheir needs directly— to indicate tlVe differ- 

ence betwebn wh.ere they would like to be and where they are now. 

Hence> this- module talks of assess incf stud ents' needs to'^^ determi nc 

program djaaired outcomes. Then by comparing the program's desired 

Q\^tcomes with *its current status (which is determined in the prjograni' s 



ma 



GUT-rent st-atUB assessment.) £he progi?am ' s .needs . can be identified, 



Th'-Vse prog,ira,m needs 'indicate^ t^^^^^ magni- 
tude of . development necessary tp achieve- "an *'ideal" program. ' 






ERLC 



13 

8 




' The ^questions on the fqllowing pages- are designed to give 
you an idea of the instructional content of this, module so you 
may determine: 1) whether you wish to continue working on the 
module; and 2) whether you want to skip certain sections of it. 
■ Each question relates to the module objective of the Scime number 
The pages of the text that present the i information required to 
answer the questions are indicated after each question.. 

Try to answer each question and check your work using the . 
answer key provided. Thisn, if you feel the module or certain 
"sections of it will be of benefit to you, contii'iue v;±th the tape 
slid.e pr^sentation\ ^ ' 



\ 1.^ Define each of the following in a sentence: . • (p; 7) 

_^ — current a b a tus^ass-QSsm Q^:^^-:-^ — — .' — • , 

Desired • outcomes • assessment : ' ■ . . - , 

Needs assessment: 

Explain- the relationship' among the three in a brief paragraph: 

2. Name three- reasons for conducting a desired _butGOin'es assess-./ 
. ment-. (pp. 19-22) . ' , . ' ' ^- , 

■ . . A. . . _ , ■ . . , • . ' ■ ■ 

:. . .B. , ■ / . 

c.~ - ■ 

3. ; There are" approximately nine dimensions 6n which desired 
outcomes assessment approaches vary. Name as many as" you can. 
(pp.- 22-27) . . - . ■ . .. . 

A. - " • . ■ " ' , . ■ . 

.- B.. ■ .■ ■ "• -, "- ... 

c. '• . - ■ * 

• D. ■ " ■ " " V . . ■ ■ '. 

■ E..' -■' • ■■ ■ , " 
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Tj r sTggT beimr~i"n~1:iTe- ±crf^--hcmd---e Glurnii'.Qrc ivo dosirod oiltcomes- 



assessment approaches. The column, at the right, contains advan- 
-fa^ a ci'es. of oo -^ h approach./ 'Match ' each approach with the advantages 
t h f? provi de^- t he ■ b est do a criptioh of. the approach, .( Pi?^v29-33y ~ — 



Approaches . 

Card Sort - 
QuestiVhnaire 

Follow-up .Study 

Critical Incident Technique' 

^"'Delphi Technique 



Advantages " 
a. 



Uses valuable obser- 
vational techniques, 
goes, into depths 
involves participants . 



b. "Gives benefit of hind-', 
sight f instruments easy 
to" develop because they 
are unusually .brief , 
, administration fairly* 
easy. 



■ . . c^. Most widely available 

• * ■ and easy to\admini ster , 

permits a large sample. 

. . . d-. Prompts- deep exploration, 

■ ; • ■ ' elicits 'fine distin-c- 

■ - tions, provid:es good 

. ^ , insights, aids in rank- 

■ " ' ^ing ^pr-iorities. . / • 

' e..- Flexible, fairly por- 

^ ^ ' . ' sonal r . provides struc- 

. ^ turo: while, allowing for!-. 

■ . ^ individual responses,'. . 

■ . gives a " ranking of needs. 

' ^ . ' '' , ' ■ ■ • ■ ^ - . ^ 

List' €he seven steps common to all desired .outcomes assessments. 
' (pp. !35-36) 

A." " . ' " \ ' -'-^ . ' .'^ ' 

C. • • ■* ^ ' ^ ^ . . / 

^b. - . . ' ' ' • ■ . ' .■ - ■ ' '•. ■ 

' ♦ - 

F. - . ■ . •■ • ' . ..: . , ^ . • ■• ■■■ 
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List, five a-c tivitieS ^nefcessary -in planning^ ancl- codrdiriating 'et 
.dQsired oatGomes assessment .v/ith advisory groups. (pp. 37I.39) . 

A, - • ■ ^ >■ . ^ ' -1. ' •; ■ - ' , ' ' ■ - ^ . " 

■ \ ' ■ ■ ' . ■ i ' . ■ ' ' 

B. \ ■ . ; ........ - .- 

\ '* ' o ■ ' • % ^ , ' ' , ■ 

c.- V.' . . ■ .• • , ■• ' . ■ _ . • • 

D.. / - 

/■■■..•.■-•■■. ■ ■ - 

e/ •.■ . - : ^ . ■.. .... . 

/■,,■"■--."■ ■ • . - ■ ■ ■ . ' ' 

|ark fhe 'statemGnf (s) below which- -should - be 'considered in ident 
'fi-fying the areas on which the desired outcomes assessment will ' 
focus.- (pp. 43 &.44) . .■ • . - 





. A, 


Defining" its- putpps.e . .. — • ■ 




B.i, 


> ■ • - 

'Considering various procedures and their, usefulness 






related to -the defined purposes. _ - " . 






Establishing decision processes and' rules^ a.nd docu- 
menting these.. . ' ' • V . • • • 7^;? 




D. 


All of the above. . ^ " - 




E. 


None of the above. ... . 


Define : 


.(pp. 48-50) ■ 


A. 


Three important considerations in selecting the sample: 




1. 






2. . 


^- . \ . , . / . • . ■ . ■■ . . , 




3. 




p. • 


Two 
1. 


alternative sampling strategies: 


C. 


2 . 

The 


procedure most appropriate .to your situation: 
* * ■■ ^ ■ 


D. 


The 


' ■ • , 

rationale for this choice: " - - ' ... 



;.;9. List: (pp, 57-61) > . 

, A. Three '"iinportd-nt- considerations "in devolopiny the-instrumonts, 
and- p*rocedure5 for a desired outcomes .assessment: • ■ 

■' . . ■." 1., ■-. -■ • ■ 

2 ■ . . " * ■ 

■ . . . 3 .. ■ \ ' ' 



■/ 



•B. The steps yoii v/ill go through . in .this- developm^enf: 
•• 1- ■■ • . ' . ■ . - 

3-. ■. ■ 

10. List: (pp. 71-79)- 

A.' Three important considerations in administering a desired 
outcomes assessment instrument. 



2. 



\ 



A 



B. . The steps you will go through in this adnrinistration; 



11. .Identify three appropriate procedures involved in summarizing- 

and analyzing collected data from a desired outcomes assessment 
instrument..^ (pp.. 85-89) 



A.. 
■B, 
C. 



\ 
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- • • Mr 


12. • 


Summarize throe important cousidcrations in applyino the 

results. of a data analysis- to a definition the, desired .out^ • ^ •* 

comes for your program: (pp.. $1-94) 






A.. ■ 








B. 














13."- 


Name: . (pp. 92-94) . ■■ ■ ^ ^ ^ ' . / ' •• ' 






A. 


Three target audiences that one. .might have for a report 
summarizing procedures and products of a desired outcomes 
assessment.. • v ' 


J." 
















- 






3". . ■ ■ ' ■ > ' ■ V ■ 


• 




B. 


Possible . topics of y.our reports . ' ' ' . ■ 
1.. - . ^ . • ■ • • 

2. '■' ^ • . • . • 






"r 


3. . _ ■ 

4. ^ ■ ■ . • - 

wciyb m wnji'Cn une repopu wpL-ii oe uaiioxeci to eacn or i 
the audiences listed in paVt A of this ^question. 

1. ■ ■ . . • • ■ ;• • 

"2. . .- ' 

3. - . . ■ - ■• . '■■ .. 




- 






«■ 






■' ' ' • "... ■ * 
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y AUDIO VISUAL INTRODUCTION - ' . , 

; v"" This mociule ; '."Assessing Desired dutcdmes/' includes ■ , 

an au^io cassette and slides 'to introduce you to this, phase ' 

of the staff ^'development series. We suggest that, you and 

^ ■ ■• ■ ' ' .. 

any othfer individuals who are cJonsidering w this v ' 

package first take a few minutes to view and hear J:hi.s- pre- - 
sentation. The tapers .playable on any casse,tte recorder • ^ - 

The slides are also^ standard and numbered in the order of 
appearanc,e. An audio oue ("beep".) indicates the poii^ts at . . 
Which you are* to advance to the next slide.. - Begin with the 
t-i.tle slide in the projector gate, - 

■ - In the event the tape.-slide is not available, you may read 

through the. 'tape-slide script which is\located in Appendix B. This 
■ . ■ ' ■ — ■ . ■ ■ \- - ■ ./ - 

Will give -you a quick overview of the contents of the module. . 





DESIRED OUTCOMES 




^RATIONALE; , - ' . 

■A fundamental question should be addressed at the 
beginning of this module'. . Why conduct desired outcomes 
' assessments at all? Your guidance, and* counse.ling program 
may be* meeting all the need^ jf youth- already, or a. sub- 
stantial proportion of ther^* • It -may be grounded- in the 
best theory available :conjerning effective programs . And 
conducting the assessment' will' ^^equire .a ^substantial amount 
of' time, money I'^and ^^nergy, it ife tp be done* w^ll. . Why 
go to all the bother?. . . - 



The answ^er to' this ^q.uestion varies from instance to 
^instance, but 'certain responses seem to apply/f airly g#n- 
eraliy. One reason relates to the gr-qwirig demand for - ; 

^« • ■■ ■ ' „ v ^ ^ ■- . . . 

accountability m, all aspecta of ^ education , and particular-^. 



ly'*in area's outside tlie ^traditional cicademi'C subjects. ' 
^ Educfators and counselors ar.e bieing told by those who pay' 

trheir salaries that they had better start.'b^ing able to ' , 
. justify • their use of the taxpayers' mon p^ . . And this just- • ' 

i*ficatian must be in concrete, measurable terms, not vague,. 

statements. So part of ^ine- answer is that practical demands 
. from thp society are forcing all programs in education to \ 

become- accountable . ' ."^ ' ■ , . . ' . 
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assess ?. 



a c count - 

ability 

desired 



TJhis explanation favors the trend tov/ard accou^itabili ty 
. • ♦ • ' ' .* . 

with negative connotations, when in fact the. r^eaaons for it 

f • ' ^ . ' * , -. . . ' ' * . 

i^te ntg)t "rtegative . . Don't counselors Qwe it to tliehrselves 

^nd their clients to know in precise and comprehensive terms 

what t'hose clients- ^eed and want? In the past-, programs 



IT- 



^trave of ten • been" structured around theories; developed by 

■ ' ' ^ / • |- " .'. • ■■ 

. scholars v/ho' work in settings' thaj: are removed from .the 

■places where theories are implemented. These theories are 

..taught to suc9ceding^ .genera tions. of counselors who have 

■'.con'jinued to use- them .iiT^olt^^-ev^^x^-c^h-a^ world; To. be. 

^i.a'fele to ^iaim they are addressing the^mpor^^ problems of. 

* the present,; counsblors need' current in'farmation>a.bout what 

clients need. and v^ant, based on those clients' own state- 

ments. Desired outcomes assessTnents suppl^ such informa-^' 

tion.- They allow programs tp have the most up-tp-date 

- ' ' ' « , . w . • . f 

■*inf ormation possible, information that . comes^ from studentjs^ 
..themselves, not theories ♦ These programs can be structul-ed 

in a way that is^ open to real accountability. ^ ' 
'^'^ - Other reasons also contribute to the case for assess- 

irig' desired outcomes..' .Programs are often structured around 

'system needs. - .Desired outcomes assessments reinfoifce the.. 

, - ■ . ■ • 
importance of youth needs." They encourage the adaptation 

.• . - . ^ • 

of the program to people, rather than vic.e versa; They 9an 

also supply very comprehensive data on these youth needs, 

touching on far more variable^ than earlier guesswork or* 

hunches;. And they usually work to- insure coordination of 

programs. If therfe'are four counselors in a school thire 
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will probably be four theories on which prpblepis should be 
addressed .first With a data base established through a 
desired- outcomes assessment , empirical evidence exists con- 
cerning the needs* and' v/ants of y^th. Tho^^four counselors 
can .reach a consensus -on priority problems apd" work toge- 
ther *t<|^ fulfill identii^ied^ needs and wants. ^ 

Still another 'benefit caft^'come .from Conducting a ' " " 
d'esired outcomes* assessment. Often programs • have existed 

for extended periods of tinie withput ever undergoing an^^ 

■** ^. ■ ■ . . ■ <■ • 

f undamentaL..iy.anninq 1" They may ha\^e feeen. started decad'fejs 

• . . " ' V • ' • V. .. <. • ■ ' ' * . * 

ago, and hav-e simply gone oh through inefrtia 'and tradition. 

A* desired outcome* as,sesSment begins ai y;hole py-ocess -of 

planning anc^ ey43iUating- that c-an be invaluable. It will 

help a program to set-;,goa:jLS "f or; itself and then gauge its 

effectiveness in terms of progress toward thosV q^oals. And^ 

rather than ending the planning and evaluation, (desired out- 

comes assessment should help to build these processes into 

• .-■ • • ^ . , ■ ■ ■ I ■ : ■ \ 

the very structure of the program. A d^psired outoomes . 

■ * ' ■ J ■'- ■ ■' ' ' 

assessment* is not fin^. It (^ncourages further-/ assessment, 
and helps to- establish a climate of flexibility! and alert- 
ness- foi: change. ' In xivis"^ v/ay the. needed structure for - 

■p 

change scan be built into programs*. . HopefuJ^ly, they will be 
less li]<^ly to become outdated by society's breakneck pace 
of change. : / ^ • ^ 

To sumniari'Ze, th<2nV there are numerous reasdns for* 
condijc-ting- desired putcomes assessments. They supply the 
means for accountability'^n guidanoje and 'counseling / 



•programs'; give concrete,, empirical data around which to 
.'structure programs; respond ±o youth"' needs - rather than 
•s-ystem netsds ; providfe comp:i-ehQnsive data instead of iso^ 
lated gu(^ss^s ;• increase program coordination; . a^d initiate 
a process of planning and evaluation designed 'to, kcjep pro- . 
grams current . ^ Of . course they achieve these purposes' in 
direc|: relation to how well they are done. The rest df 
this module will ba addressed to the "how" pf desired out- 
•CQm^3s assessm(^ts . . < ^ ' . . • . 

DIMENSIONS ON WHICH APPR^^^^ V ^ ^ Z • 

• The pl^rase V'desired outcomes assessment" is a gejieric. 
one. It does not define a particular method or • technique , 
merel'y the general proc^ess of assessing tl^e goals yQ<}th 
want to adhieVe . tl^r^u^h a program. When one begins to d^etvX 
into particular techniques available for this process, lie : 
soon realizes the subject, is cbmplefx-. There are .nume'rous \- 
dimensions on which approaches vary, and many different 
techniques. that have actually been tried. .A useful task 
for the program planner v;ho mi^st design a desired outcomes' 
assessment is to understand the more ' important of these ■ 
dimensions, arid the principal techniques that have actually > 
been commonly used. For 'this^ purpose,- the module will 
brief ly cJisc'usS. nine dimensions oh xyhich desired outcoriies 

Approaches vary, and then^^rdescribe f i^ve examples of the more 

" - • • . ■• • . • , • ■ • \ • 
commonly used technique s. ,0 ; . . ^ . 

A. : Direction and^'CQntent of Focus 

Depending on the^urpo.se of the assessment, it may 




focus on various -contGnt^ areas . ' Basically the variable hero' 

■ ■ .. • . '> ' . ' 

±s, what, is the content- of the asbes.sment? Is it lookincj 

• ■ - ' ' , ■ • • ■ ■ . ■ . •• ' ' ■ 

for \inf(5xmation on family situac jns or on school situa- 

tions?' \ Is.- it seeking information on interpersonal needs ^ or 

on acad/cmic needs? Another partifcularly important dimension 

is whether the assessment* has a problem orientation or ''a 

solution /orientation. The Mooney . Problem Checklist , for 

■ example , ask« respondents tp indicate from a series , of lists 

the problems which are applicable to/ them. Then it- tries to 

idehtif y v/eakne.sses and faults. Contrastingly, the Prior- 

ity Counseling Siitrvey generally asks respondents to indi- ,^ 

bate. those things^they like or are good at, thereby defining 

positives'- instead of negatives. . One might say it has a sol- 

ution focus, as opposed,' to a problem *focus. Eacji focus of 

■' ' ' _■ ■ 

course has advantages , ^ and both ultimately help to .efine a 

program's direction. The better choice depends on the pur- 

pose of the assessment. In any case, direction and content 

of focus are important dimensions on which desired outcomes 

assessments vary .. . , 

B'.^ , Breadth of Focus ^. ■ ' , 

"/This referiS to how comprehensive and broad ranging the 

assessment is to be, and relates 'to how many content areas 

/it covers. Do ypu^want to probe into one or two isolated 
area's^ or cover every aspects of the respondent's ' lives? It 
is possible to" design assessments that have a single focus, 

^tbat focus on several areas, or that are very comprehensive 



second 
dimension 
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in - their • focus . 'The narrower the focus, the more that can 
be learned about" any one area. The broader the focus, the _ 
better the oveifall pic ture , ' but- the weaker the understanding 
of any one area. Again, the choice depends on the purpose ■ 
of the assessment. ^ . 

C. Depth of Coverage ' 

. . < . ... 

This is related to the last dimension, but' refers in 

particular to the overall length of the assessment. Is it 

■ ^ ■ ■ ■ ' = ■ 

^ . .. ■ 

to be a ten minute checklis t, or a three hour interview? 

Will you probe deeply. into the area you decide-to focus on, 

vor ease your data analysis task --^by using a few key items in 

^ . ■ ' " ' ff ■ ' . " 

a given area? Do you have the resources'^ of time and money 

* *• ■ ^ 

to undertake an extremely thorough assessment? Will you be 
content if the information gathered fails to go into depth 
,.and provide fundamental direction? These are questions that 
mus t ^ be ■ ansv/er.ed ill deciding on your depth of coverage . . 

D. "Degree of Structure 

The question here is whether you will - determine the, 
nature *of the responses by asking the respondent to check, 
circle,- or rank, or whethq^ you -will ask him to structure, 
his own response on a more open-ended questionnaire or in 
an interview. The former choice makes the taskiof analy- 
zing; the data considerably easier, and some structuring 
will probably always be 'necessary to allow .meaningful sum^ 
mary and analysis. On the other hand, more open-ended ■ ' 
questions . allow more individual expression and fuller' grad- 
ations of feeling. They -may also produce insights and 



suggested • solutions that s true turou formats f£iil to elicit. 
The benefits and v;eaknG*saes of each degree of structure must 
.be weighed and a: choice made^to conform y/ith the conditions 
pertaining in a, given instance, 
E . Personal j^zat ion 

Assessments can range all J:he way from the individual 

.interview, in which ^the respondent talks .to an interviewer 

and has his thoughts and responses recorded on a tape or 

written down, to the mass inventory in which thousands of 

\ inciividuals indicate their feeling b^ marking a paper. In 

between is a w4iole spectrum of approaches which lie some- 

vmere along the line defined by these two extremes^ ' Degree 

of personalization is thus another dimension on which ap- 

pi^baches vary, and^^nofcher variable for the planner to con-- 

\ ' . ■ ■' ■ ■ ■ 

si^er in choosing his assessment procedure. 

- • - . .. ; . ■ :^ _ ; / 1 • 

\ F. Sampling Considerations . ' 

(\ Entire books h^ve been written on this' subject alone. 
Dd^.you want information from ' elementary school .students , or 
gdaduktes now out working? From siibgroups wiizhdn the stu- 
dent population. (minorities, residents of a geographical 
area)?\ From adults, and various adult subpopulations * - 

(parentis,, teachers, counselors, administrators:, community. 
re{j)reselfitatives) ? Will you sample .atl groups to be affected 
the dssessment., or only those most directly affected? 
1 youl include all students? A percentage of all students? 
StUiJentslat selected grade levels? Or a percentage of . stu- 
dents at each selected grade level? .Will you include all 
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./schools within the district, or pick one or Lwo as repro- 

' i * * 

scntative? . llow v/ill you assure a representative sample if 

■ ' ■ ' ' ' ■ ' 

you can't include everyone? The .questions " in this realm- are 
•endless, and the answers are critical- for reliable iresults. 
The training section of this module goes into the sampli-ng^ - 
question in some detail, offering suggestions and guidelines. 
It- is a central dimension to consider in conducting a 
desired outcomes assessment. 

G. Quality . 
Whole books have been writt^rj on this subject alone. 

. Quality of the assessment instruments and pro^cedure itself 
-is a subject so broad that dozens of variables contribute. 
Four key oneb, however, must be cons^idered under 'any cir- 
cumstances. These are. validity ^ relial^ility , generaliz-. . 

■ ability, and replicability . 'Validity refers to whether an 
instrument actually measures whkt it is. designed to. Reli-^ 
ability refers to whether an instrument produces consistent 
results over repeated Usage. . Generalizability refers to 
whether the results obtained . with a limited sample are 
accurate for the broader group they are intended to repre- 
sent. . Replicability refers to whether the entire process- 
can be documented and rQple.ated elsewhere. If the desired- 
outcomes assessment is to be a useful one, it should meet 
high ' standards on' these criteria. 
II. Availability and Costs 
An obvious practical consideration relates to how avail- 
able a particular technique is; that is, whether instruments 
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and procedures arc already in existence , whether they may 
be purchased commercial ly% or whether they must be written 
and" developed. And of those techniques available, what are 
. their relative' 'costs? Cpuld a technique be applied more 
cheaply by developing one ' s own materials* and procedures? 
The practical necessity of deciding on a technique that 
your district or school can afford is one that must be con- 
sidered. 

• I • Ease of Admin i s t r a t i o n * • 

Another' practical* dimension on which desired outcomes, 
assessment techniques vary .is that of ease of administra- 
tion. ^ This includes several factors: ease in processing • 
data from the instrument to produce usable summaries; time 
of administration; amount and nature of training necessary 
for administration; s'cheduling flexibility of administra- 
tion^ to adjust to school schedules and available . space ; and 
apprgpriateness of the instrument' for the level of respond- 
ents participating in the assessment. Appropriateness of 
the instrument and procedures to the' purpose of - the 'assess- 
ment-is another factor which Contributes ■ to ease of adminis- 

' * . ^ 

tration. Practical considerations can't be. ignored, and^ 

^. ' * ■ . .» - 

ease of administration i6 a central one.'. 



PROGRESS CHECK \ 

Write a phrase which gives the defining character-is tics of. 
each of the Eollowi^ig deminsions on v;hich* desired outcomes assess- 
ment approaches vary. 

Direction and conitent' of focus: ' ■ ■ ' - ■ ' ' ' ■ ^ 

(p. 22) _ 
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Breadth of ' Focus: 
(P- 23) 



Depth of Coverage 
(p. 24) • • 



Degree of Structure: 
■(p. 24) 



Personalisation : 
(p. 25) ^ . 



Sampling Considerations: • • ■ 
(p. 25) 

I ■ , , •• -- — 'i— - - . . ■ -, 



Quality : . 
(p. 26) 



Availability and Costs 
(p. 26) 



Ease of Administration; 
(P.- 27) ' / V 



if you are unsure of the definition of any of tliese dj-mensions, 
check back over the preceding pages. • 



PRACTICAL EXAMPLES „ - 

All of the . nine diraen^ions havq been discussed on a ^' 
fairly theoretical, abstract level. What are the •techniques 
actyially in common use today? How do they combine the dimen 
slons into, a workable format? The follow' ng sketches of 
five actual techniques give brief descriptions of common 
realistic alternatives for conducting desired outcomes 
assessments. . v 

The Card Sort 

In: this technique an individual is given a "deck" of 

cards, each with a st-atemfent relating to. a need he might 

have. For exainple , on^ card'might say/ "I need' to improve 

my • understcinding ' of what I read." A deck- of cards includes 

statements related to a common area. Individuals sort 

~th^oug4v-each-deck they receive, eliminate those cards v;hich 

do not represent needs- for them, and arrange those that do 

in order from most impor^tant to least. The cards selected 

from severcil deci;5 may then be combined and arranged by the 

individual according'to their, importance. /When data on the 

needs of any individuals are combined, the areas of highest 

need, and the highest needs within each-area are established 

From high and low ranking' needs, program objectives niay be 

^■^ . ■ . ' , 

drafted -to give dir>ection to program developments. 

I ■ . ; ■ - ■ ■ ^- ■ . 

The'card sort boasts a number of strong features. It 

is fairly. personal, allows for flexibility in breadth of 

focus and depth ofo^coveirage , and provides structure while 

«• * * . . • ' ■' ' 

.allowing for individual responses (through individuals addin 



their own cards). It is less v;oll knov/n than th.G><nucstion- 
naire, .but not . cliff ictilt to develop'. Administration xs also 
more complex than that forMiiost questionnaires^ but not 

7. ■ ■ ■ ■ 

necessa:^ily prohibitively so. ' . / 

'Slje -Questionnaire 

What is meant by this term is a paper and pencil in^tru 
merit ^,to which the individual responds in any of a variety of 
ways'. These include written responses to op^en-ended ques- 
tions, checking yes or 'no, checking one of several multiple-, 
choice responses, ranking a series of statements .from high- 
est to lowenst, and rating, statements along- a. numeifical or 

■ . \ . _ ■ ■ • 

written scale. Some examples of conunercially available 

instruments questionnaires are the Educational Needs Assess^ 

ment (a rating format) , The Mboney Problem Checklist 

(multiple choice checklist, followed by a ranking procedure) 

^ ... 

and the Priority Counseling Survey (multiple choice for- 
mat). Ouestionn^ires are often for a particular asse'ssment, 

especially if open-tended responses are desired." 

^ • •. ■ ■ . . ^ 

Questionnaires represent the most common form of 

assessment, because of their ease^ of *administrxition and 

availability. They can also be fle>::'lbie in terms of breadth 

Of focus, and often permit a large sample. because of their 

ease in^ administration^ They usually 'provide less depth of 

coverage and involve participants less, fully in the assess-- 

ment process ^than the card sort,, and also lack, the latter's 

personalization.' .. 
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- TliG Follav/-Up Study . ' • 

This is anothei:.. co^mmon form o£ assessment,, character- 
ized moist distinctly by the sample, of .respondents. , It is' a 
survey of individuals now removed from a program, such as 
graduates from high schoolvwho are asked for - their ^thouyhts 
' on the^ high school guidance and counseling program, A fair- 
. ly short, structured or open-ended questio/inaire is fre- 
quently employed ' in follow-up studies, although an inter- 
view format is. sometimes used. Mailing strategies often p.lay 
jan' important role in this kind of .survey ,• and achieving a 
-iMfiigh level- of response is most important. . 

• The follow^rup survey is probably the most limited tech- 
nique'in common use. - It fails to sample'present or filturi^ ' 
clients 6f a program,^,, but does- boast the advantage of ques- 
tionin<5, respondents who have, the benefit of hindsight. 
Depth of cov^r^ge *is nofe 'often achieved v/hen mailing strat-' 
egies af e used., since a-- lengthy instrument is generally asso- 
' ciategl with loV response ^rates . Pollow:-up studies usua3,ly 

employ individually- tailored ihstrumeri*tsj sisice these can be 

^ • ^ ■ ''-^ ' • • 

designed to pertain, directly to the situation being ^'folr 

iDwed-up. — \ \ ■ ■ • 

The Critical Ihcia^nt Technique^ ' 

Critical incidents arse example^ of observable behavior 
■with .a cl^ar intent or purpoVe and at least one observable 
outcome. They were used at leaM: once, in a projectx in 
Bucks County, Pennsylvania,, as part of a desired outcomes 
assessment. Ten global goals fot th^educational system, logd 
.been identified,*' but they required further definition 
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fourth 
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because they were very ^abstract; - .Thousands of .critical ' * 
incidents relating to the :goals were collected from parents, 

._tG-a^4^e^r— *ind students/ .allowing precise behavioral deflni-- - 

■ ■ . . . ' ' f\ ■ ■ • 

tions of the goals ^.both; in terms- of student behaviors and 
teacher strategies to encourage such'' behaviors. A; Class- 

ification scheme, was thdn built for. each goal, and'.a series 

^ ; , ' \ '.^ V ^ ^ . y " ' 

of "needs-assessment" qu<^stionnaires were developed. The 

. - ■ - /I, 

' ■ ■• . ^ 

questionnaires v/ere used to determine'''' t-h.^ -je^tent to which 
appropriate student behaviors were being .deii(orist rated. "The 
critical incident technique served/to define the'"desired 
outcomes of. the educational system. Once this wais dbn'G> - 
the: current status of students, and thus of the educational 
sy-stem, relative to these desired outcomes could be asses.sed. 
'JDhis process could be used with guidance and counselinl^ - 
programs as well. . _ ' 

This is an unusual form of assessment, but tiie obser- 
vational. techniques * involved are a valuable tool. Th^ pro-. 



cess" certainly goes into depth, and requires training in - ~ 
Observation, extensive collection' of incidents, and S'krll in 
anaiysis^-— Jit. involves participants in the ass'essmerit pro- - 
cess, providing personalization, and can be flexible .in its 
focus. It may be challenged because it relies on predefined 
goals, rather than seeking the perceptions of students and 



others. This challenge is not serious,, however. Any struc- 
tureiu instrument must rely upon predefined goals to some 
degree. As long as .the goals are sufficiently global, no 



participant will be • constrained in his responses/ 



\ 
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The Dbl^)hi. Technique ' . . ' .. ^ . ' ■ .. 

- -. This is a irtaans of siinplifylny the task of identifying 
and ranking needs and ' priorities , ^^and arriving at goal 
definition. It- refers to a series of intensive inta^rroga- 
tions^of samples "of i^idividuals ' (most' frequently, experts) 
by means of mailed questionnaires. The mailings are inter- 
spersed with controlled feedback to the participants. 
Responses •to*'' each mailing are summarized and returned to 
respondents anonymously. Competing opinions usually con- 
verge and diverse opiriiona, are of ten blended into— <4is tine t 
and clearly gtated majority and " minority: opinions . Person^ 

- - ■ . y * • ' '■' ; ; • ^ ^ " , . ■ ^ • . ■ 

a'lity conflicts are usually avoided, ]^ \ 

. « • — .. " ' 

Weaknessels of the Delphi \technique^i'nclude the fact 

V . . . 

that.it usualiy soifnples only selected^xperts, tH'at it 
v^re quire Sr much tabulation, . record keeping, and mailing , ^ and 
that 'it^*proy'i4e3 consensus, but not necessarily ;tho "best" 
'JiJidgment. On thd other hand, it of ten .pi^ompts deep e>rplor'- 
. atioit and elicits : fine perpep^tions and di3tinctions ,1 gener- 
ates lively interest and real insight, and ai^ds in identi- 
fying apd ranking rfeeds- and priorities. 

... ( ■ ' ■ V ■ ' 

PROGRESS CHECK . ' 

In a sentence^ each t ,g,ive the defining characteristics of the 
following, examples of assessment techniques. %hen list" an advantage 
and disadvantage for 'each. ' 

I.' The Card Sort: . > 

29) : ' ~r~' ~ ' ~~ — 



Advantage : 



Disadvantage : 
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2. The Ques tioAnairc 
-■(p. 3 0) 



Advantaged 



Disadvantage : 



• The Fqllpw-Up Study: 
Cp.' .31) ' 



. Advantage ; 



"disadvantage ; 



-4'.' The Critical Pncident Techhlaue: 
^ (p.^ 31) V> 



Advantage 



Disadvantage : 



5V The Delphi Technique: 
(P.^ .33) 



Advantage ; 



Disadvantage : 



If you are unsure of any of your ansv/ers, checJ^ the. descripcior 
-of the techniques * found on the preceding pages. 
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INTRODUCTION • . \, 

It would bo. desirable , to include extensive training - 
in all 'of the techniques discussed in the previous section, 
..but such a cQursc.is not practical.. WhiJ.e each ' technique 
boasts advantages and each has disadvantages, a number of 
factors suggest the card sort as the ideal metlijDd. Because 
of its f lexibilit;J^ , ^it .can be adapted to almost any situa-- 
tion or need. It^produces .a rank ordering of needs arouhd . 
which to^ structure programs , and this is a most useful form 
of information'. The card sqrt technique allows for .real • , 
•comprehensivenes.s. , It .uses 'a bomputer to aid in summariz- 
ing the- data colTected.^ It is a we,ll' documented and tested 
proqess&r while also progressive ' and innovat:jjVa as compared- 
with questionnaires and follow-up studies. Technical assis- 
tance is available for tliose who ijeed help in ,its use. For 
these reasons, this module will concentrate on . the card sort 
method of conducting desired outcomes assessments. 

The assessment process may be broken down into a numbi^r 
of stfeps, which pertain Whatever the particylar method ; 

chosen. They are as follows: - 

•■ ► ' . \^ . 

1. Plan and coordinate with the advisory group(s). 

* ' ■ . ■ ff 

I 2. Identify the areas on which the assessment will 

• focus . 

' • ■■ 
3.1 Define and ; select* the sample. ' 

4. Develop the instruments . ' r 

. ;~ ; ' • ^ ,35 . ' [ / 



card sort 

most 
desirable 



assessment, 
steps 



5. • Admi/iister the' instruments'^. " - 

' ' 6. Summarize and anaiyze the data: 

I • ■ ; . ■. . ■ .. . . . " ' . 

7. Apply the results to a definition Qf the desired 
outcomes . • . - 

\ The section of the module 'whicH7 follov;s will .detail,. 

these, steps for the card sort fi|:oce4ure.,. presenting cohsi- 

•derations, guidelines, and Suggestions for each. In .steps 

four and five./ the ques tionnaia^e ' an4 'follow-^up study tech- 

■ ni^ques will also be touched on briejly. The other sections/ 

dn large part apply , to all methods equally well. The steps 

will- i),^ discussed iii terms of the card sort, but individual 

aL^dfainistrators may. generalize them' to another process if ^ 

they wish. A Procedural Checklist for each level may be 



foUndf in Appendix C .. 
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List the Iseye-n steps 'of 
techniques. Xp^- 35) ■ 


the assessment p^oces^- which apply to* all 


1. . ' ' 


■ - / 




4.: . ' - ■ 


■■ ■ i ^ - - ■■. ■ 


5. 


' . •■ A . ' ^. 

. ■ „ ■ / ■ ." " " . . • 

— / — ' — \ ^ 


6.." ■ .' ' 


/ ■ ' . - ■ 

■ ■ A • ; . •• ' ' 


7. . . 






• ^ -• ■/ • . . • ' 



If yolli are. unsure'' of .any of thefee, check, the list on the. preq'eding 
page • . 



PLANNING AND COORDINATIN^^^^^^^^^ . 

The first task in conducting a desired outcomes assess- 
ment, is establishing the needi f or such an a§.sessment in the 



one 



minds of those vho have the power to approve it and put the 

re.-sultS'Of the assessment, to use. Enthusiastic cooperation 

and support from such a body can make a huge difference in' 

the success of the assessment effort. Making tlie case for 

.the. ass.essmentV should not l?e difficult if information is • 

gathered to . illustrate the lack of direction programs are 

apt . to have without concrete data on /what students need -from 

1 . ' • 

a guidance, counseling, .arid placement program'. Every stu- 

i'. " . •*. ' 
dent stands to be^nefit from participati^ng in a program which 

has been structured to meet his needs and the needs of other^ 

like him,,, rather than a program based on vague goals that 

. ac^ministrators or counselors have established in the past . 

> ■ ■ 

;But ±hose with "the power to improve guidance, counseling, 
. and placement: programs must be given a chance to understand 
this point. Thus, the first big job in conducting an effec- 
tive desired outcomes assessment is one of communication. 

Closely -related to the communication task is the ne.ed 
to insure the parti6ipa tion of all other groups to be 
^ affected by the assessment in establishing the goals and 
process of the ^effort. Counselors, tea<j:hers, administrators, 
students, parents, and ci®iTimunity representatives will sup'-K 
: port a worthwhile project if they understand its purpose and 
1 the benefits that .wi 1.1 result from it* But if " counselors , 
j for example,' are forced to particij^ate in a process's they s^e 
las threatening to their jobs anpl the way they have been 
] doing theni,' without being given aii understanding of the pur- 
•poses of the assessment apd the kiijd of positive . change for 
themselves asl well as students*' that is likely to come from 
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■ itpv they will be unv;illincj partner's in the effort. If they 
iacV' enthusiasm, no doubt teachers v;ill also. Teachers tend 
to convey attitudes to students, students to parents, and 
^ sooii a failure to convey the -purpose, importance, and proce- 
dures of . a worthwhile effort to one key group may sabotage 
the whoi^j operation. This point, cannot be s1:ressed toe 
strongly: all parties to a successful assessment of the 
.desired outcomes of a guidance, counseling, and placement • 
program must be brought on board and . kept enthusiastically 
ijivolved throughout. Communication is^ a key. 

One of the be"^st'ways t-o assure full understanding and 
communication is to document . the purposes of your effort and 
the means you' plan to use in achieving ttose purposes, and 
then to distribute a brochure or pamphlet containing this 
information* On the next page is ■ such a brochure. It out- 
lines the essentials of the assessiment project, and puts t;he 
effort clearly in focus. 

Certain other considerations require a^tenti^^at this 
initial phase of an assessment. One is to. choose thh type 
of instruments ^rid procedures that will used. The vari- 
a.bles and examples described and discussed in the first seg-. 
ment of this package should prove useful to the advisory, 
group at this point.. The group should answer questions such 
as: which approach fits best with local restraints and 
needs? What are the conceptual variables mos,t important , 
and the pragmatic' factiors most significant? Success, or/ 
fk:j.lure may" hing.e on the care v/ith which choices at^ this 
stage are made. : : - ' , 
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Following these decisions . arc thoso^ related to'plan- 
ning for the subsequent steps, in the assessment process- 
Choices as to who will ,carry out each such -step are key in 
assuring th(^ .success of an assessment. * Who is best quali- 
fied to decide the areas on which the assessment v;ill focus? 
'Does an expert in sampling nee9 to be consultcil to help in 

the. definition of the sample? Who cain best choose an 

i . ■ • • • ■ • • 

appropriate. 'instrument or develop one specially .^tailorad to 
local needs? Wh9 should be in charge of carrying out the ^ 
administration of the instrument, and how much help will he 
need? Who will analyze the data so that it can be used 
effectively in determining the desired Outcomes? Who must 
cooperate at each stage of this process, and how can lines 
of. communication be established so that the necessary infor- 

mation will flow to those who need it? Answ^^ring these 

. - - ' " ] ' ' ^ 

questions and making, decisions determining responsibilities' 

and heeds at each phase of the assessment is a job^tfiat 

falls on the shoulders of t,he planner,* -in cooperation with 

the advisory group. The fuller the planning and "articula-- 

tion.at this stage the more satisfactory will l?e the out-- 

come of the assessment. 




> 



More specific considerations sshould be ^^dressed in 
planning the desired outcomes aissessment. 6ne is the coor-r- 
dination of this phase of program, development with preceding 
and succeeding phases. T.his coordination includes several 

dimensions.- The first is time. It ' s inappropriate .to 

*» ■ ■ ' - 

complete the desired outcome's assessment in September,-, for 
example, aind npt have the results of the current • status 
assessment available until June, since^ defining "program 
needs requires a comparison of the results from the two- 
'as'sessments. Also, it's important" to plan the two assess- , 
ments so that the data from each is comparable. This per- 
mits .meaningful discrepancies to.be determined. Making sure 
schedules mesh- and data are comparable is largely a function 
of. the effectiveness of initial planning.. 

Assuring meaningful data is another major considera--, 

. - . ... , [ ... 

tion for this phase. One must design the desired outcomes 
assessment in such, a way that its i^esults haye practical 
use. in the implementati'on of new programs. 1^ it ' s outside 
the- practical limits of the district * s budget to establish 
a career education center, complete with library, librarian,, 
coihputer linkage to a national^ center., study booths, films, 
regularly scheduled fi^eld trips to a variety of job sites, 
and 'lectures by visiting experts in the^'career education 
field, then it ' s pointless to. ask students whether they need 
these things. One can a^k questions which determine a cjon" " 
eral need in a way that permits response within the con- 
straints of available local resources . This i-s another 
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example of hov/ coordination of the desired putcdmes QS SGS S — 
ment with previous and subsequent stops is essential in the 
initial planning phase. • i 

Another issue that requires early'; planning and coor- 
dination is the determination ot the final form of the 
assessment data/ Too often, masses of data have been col- 
lected in an efficient and sensible manner, but procedures' 
for tabulating and summarizing them were not cons;idered, 
wiien a computer, is to be used for this purpose, it is only, 
reasonable to consult a prografruner; In smaller scale 
ef torts in which clerks are to accomplish this task, ini- 
tial planning for tabulating and summarizing data is just 
as important. ■ 

Planning and coordinating a desired outcomes assess- 
ment with advisory groups requires that many issues be 
taken into consideration in order that numerous decisions 
can be made. In this step, as in all phases of a desired 
outcomes assessment and the larger effort to which des4.red 
outcomes assessments contribute, it is important to docu- ^ 
ment each decision and the reasons for it. A precise state- 
ment of procedures and the rationale for them is, valuable 
for several reasons. Such a 'statement will facilitate cqjii- 
muriicatibns** and understandingf during the course* of pr9gram 
development, implementation and. evaluation. It maty help 

keep' a project true to Its "original intent and perhaps 

■ j> ■" ■ 

assist in answering questions which arise as plans are 

■ ■ ' * i * . ■■ 

implemented. Finally^ a continuous record of program devel- 
opment will ease the task of preparing formal reports on the 



-project. .It, is rccommeQclGd" that "v/ritc it down*' become a 
guiding principle of the prograip development effort. * /' 

* IDENTIFYING THE AREAS ON WHICH THE ASSESSMENT WILL FOCUS 

- . ' This .'is tlie step in the desired outcomes assessment pro- 
cess after planning ahd coordinating . with advisory groups./;- 
Once people, have been brought on board in the effort, and - 
initial decisions about who will conduct the assessment * and 
v/hat route it v/ill take have- been made, a precise focus must 
be defined for the effort. • What is the central purpose of 
. the assessment? Is it 4o gather information on one or two 
. keyoareas, or to estaiiLish a basis for a comprehensive pro-^ 
gram? lias a particular problem sprung up recently, such as 
■ drug abuse, racial pension,' or a high dropau't rate, that^' 
suggests the ne<^d for new information on what students need 
and want, or is the assessment effort directed at the activ-" j 
ities and .objectives that have constituted the guidance, 
• counseling, and placement program for the 'last 30. years? Is 
a new prc3gr.am being considered for a particular subpopula- ' 
tion of students — those emotionally handicapped, culturally 
.deprived^ or of a particular minority group, for example— 
;which suggests that i an assessment ' is needed to focus on the 
needs of-lhis subgroup? One can define a single narrow 
focus for an assessment , try to. include. all needs students 
/might have, or focus anywhere in between, dependinci on the 
purpose o£ an assessment in -a given situation. - The impor- 
tant point is to * decide on the purpose ahead of time, and 
. structure' tXe assessment accordingly. Of course, this 



step two 
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purpose should be precisely stated and documented in ade- 
quate detail, . " i^'' ^ - . ' ■ 

The card-sort is a fliexibl6 technique' which can be" 
adapted to either broad or narrpw focuses. For example, 
the six career areas- defined in the Orientation Module- of 
this series o:& staff development packages may be used separ- 
ately , or "all together , depending on the focus desired for 
•the ' assessment i Taken together, the six areas cover vir- 
tually all. student needs and hence, possible desired\but- 
comes^ f or a; guidance, counseling,, and placement program. To 
review, the six c?ar>eer areas are: (1) vocational planning; 
(2) feducational planning; (3) learning how to learn; 
(.4) personal and social developmdntr (5) social resporlsi- - 
bility; ( 6)- leisure time use. — ^- 

— ~ ~«SF» •' c s 

Reaching a "satisfa<;tory definition of the focus of a 

* ' " ■ . ' ^ . 

dosired. outcome assessment- can best be accomplished through 

«'.- . * ' ' • 

establishing a decision process and 'rules for making- the* 
necessary choices. Sample guidelines might include: alv/ays 
. select the focus which will be the^.most cost efficient , or.„, 
the most unbiased toward a particular student subgroup, or 
the most acceptable to the community , or the most respbn- ^ 
sive to a. particular local problem or consid**eration., and- so 
on. By defining the decision process and rules, and by 
employing them. consistently , charges of personal bias or 

■ 9 ■ ^ . ^ ■ 

unfairness' can be avoided, and a more satisfactory desired 

■ ■ . . ■ . ' y ' ' ■ ■ . 

outcomes assessment can -be completedl.. . •. 
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PROGRESS CHECK; 



Communication is a key to planning and coordineiting with ad 
sory groups, the firiSt phase of the desired outcomes assessment . 
List six groups ^ that should be . informed of thq goals and pcoce;;- 
dures of the- assessment as early as possible.' (p.'37) 



A. 

B. /. 
C. 



D. 
F. 



In beginning an asse^ssmcnt process, it is best t5: (p. 37)* 



A. 
D. 



.Avoid excessive publicityv notifying only key personnel 
P'lan and coordinate with students and parents first 
Plan and coordinate :With. advisory groups'^ first 
Arrange for a radio interview to disseminate information 



A number/ of planning activities ghould .be. accomplished in the' 
initial step of \^esircd outcomes ^assessment . Check the ones ' 
below that- were emphasized' in the.. text. (p. 38) - 



A. 

"b. 

"E. 
>. 

"li. 
'l. 



Choose the type of instruments and procedures- most appropriate 
Select "1:he type of sample* * . 
Develop high quality instruments ' ' 

Select personnel appropriate to each phase of the operation* • 
Traiin interviev/ers to carry out the card-sort procedure ' 
CooSftinate current status and desired outcomes assessment \ 
Assure_ meailingful data r . • ' * \ : 

Plan or^ientation sessions 'for 'students and adults 
Consider computer needs ' ^ * ' ^ < 

Write sample items, and. pilot test these 
Develop a schedule for administration of the.'instrument 
Develop information dissemination strategies „ 



In deciding on the focus for a desired outcomes assessment one 
should consider most carefully: (p. 43)* \ 

^A.. Icjiosyncracies at Various school sites 

B. Ethnic minorities and their particular needs and problems 

C. Adult needs as v^rell as youth needs * 
D. '..The central purpose, of the assessment 



.5. The purpose of establishing decision rules for ' choosing .'areas 
. ■ • on, which an assessment will- focus is: (p. 44) . - 

- : ^A. To provide consistent- gu-icielines • * 

' - > B. To avoid charges of bias ■ - 

C. To make the decision- rationales public /: \' 

D. All of the above " . . . . ' . . . = ^ ^ 



ANSV^ERS ■ ■ 

" ' a 's 

' .a -'v 





TIVITY 



be considered, 



If x)ossiblG, meet with 
several of your fellow 
planners/ counselors .■ Assign 
them the lollowing roles: " a. 
supportive school board^ mem- 
ber; a conservative/skeptical^ 
board member;, the district 
superintendent; a school 
principal; and a teacher who • 
. . ;''^i5 hard-nosed realist. ' 

Imagine t^iey are. an advisory 
group to v/ho|Ti you are propose 
ing the idea oE conducting a 
y desired outcomes assessment^ 
and that much of ybur' support 
will depend-c on whether you 
. can provide a clear - pioturta - 
of the advantages that accrue 
from such an 'effort. You must 
also, clearly describe the ini- 
. \i ' r . * tial planning issues tliat. must. 

Orient the discussion to the following specific topics:- 




1. 
2. 

3. 

4. 

5. 



The advantages of conducting a desired' outcomes, assessjnent. 
The advantages of the card sort approach (or another approach, 
if yod prefer)-. * . .1 

What local, personnel will be interested and. availabl'e, to help 
with the various phases o'f »the pro ject. ; ' 

l^hht will be the most meaningful fori^i in. which p6 cj^llect 
information. ^ ^ ^ - 

What communicatioiv strategies will be a|vaij.al5le to assure 
hece.ssary^ dissemination of inf ormati-en^atall phases. 



Discuss^'each topic for pJerhaps 10 to 15 miiiutes, letting each 
point of view be represented in the discussion. Jot down important'" 
points of contention in each area, and' study th^se. They should pro- 
vide a good previev/ of problems to be. alert fori and issues to be 
^ihformed on, as your assessment progresses . f 



DEFINING ARID SELECTING THE SAMPLE - 1 

The ideal population to sartiple in a desired outcomes, 
assessment is everyone who will be affected in 4ny way by 
the possible ojil|:comes. ' I'f a School, distr^^ct' s guidance and 

■■■■ .i-';. ■ 52 -.^ ■■■■ , 

\ ■ . ■ . • . . . . 



^ t ep 
three 



counseling program is conducting the QSScs.sniGnt , this means 

* 

all.> students / parents, teachers, counselors, administrators, 
and corumunity representatives in- the. district. ,In occa- 
sional ^circumstances this ideal may be approached. • If the 
assessment involves, one sm^ll 'school in an isolated commuh- 
ity , for example^^it may be possible- to include ^all these . 
individuals in the measurement. This is not of ten the case 
however, and one is usually forced to adopt a strategy to 
select representatives from the total population. The key 
to success, .in this process is to chpose -a sampling procedur^e 
which insures that, thd few who are measured reflect accur- 
-ately the. will of all. ^ • 

. It is not easy to choose a representative san^Xe. Many 

factors tend to work as biasing agents. Students^ at ^one 
scRool may be brighter or duller than those at. another, or 
may have a different sefof problems, perhaps related to 
tHeir differepit location* Differences of only a year or two 
in age may cause substantial variations in jthe needs students 
Save. To account for these variations, different methods of 
representative sampling have been developed, to ensure fair- 
ness. Probably the simplest and most straightforward is 
simple random sampling. This involves tri^ng to see that 



only X^anao determines what individuals are included in the 
samp-]?fe.' It is particularly appropriate where the possible 
biasing factors are little' known or understood. Usually one 
■''establishes a list of all individuals in the largest popula- 
tion, and thea randomly enters. th^ list and picks every ^third 
or fifth (or whatever number is abpropriate) individual to 



random 
sampling 



f"' thei e will be .. 
chance differ- 



generate the -sample. While this is technically only a quasi- 
randoilii method; it is practical and adequate for your pur- 
poses l^ere- . ' ^ 

If biasing factors are knowji/ a better means to ov^r- 
""come their effect is stratified sampling. This^involves 

■ selecting .certaiiy factors, such as age, sex, grade level/ 
ethnic identity, or residence location, and deliberately, 
balancing these factors ih the. sample. If done, carefully 
in a situation where there\are important biasing factors, ^ 

. this is' the most ecanomicalVmethod , as .itallows inferences 
to be made, accurately from a\smaller numb.er of cases,. 

, * .No matter whaty^the method of sampling 
some sampling error. There will always be 

eaces between the selected individuals ' and the entire P9pu- 

lation. . With a very small sampl-e, the ' error wi'^ll be laafge. 

As the sample ^ize increases, the error shrinks. As 'a 

xough guideline, the error is inversely proportional to the 

. ' . ' . ■■ . . : ■ 1 . 

square root of 'the number of measurckiuents. In more concrete 

teri^s, it is best to incl-ude at least '10% to, 15% of the tar- 
get group in the assessment to assure adequate representa- 
tioh. * ' ' ' . 

■ Another imn^ortant consideration involves the question 
of what subgrqiaps to include -in the sampJte. Students cur- 
irenfely enrolled /Ln the school make up the first, obvious 
group. But' what age group of .ftudents should be ' sam|3led? 
And should the sample bes chosen iby ' grade level, age level, 
or school? Is it also, v/orthwhi-le to gather information from 
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formier stUclGnts, as they' lu^ve \the benefit of hindsight? 

>. " ■'. -l ■ < ■ . ■ 

They could indicate those school services which contributed 

■ • ' ■ • ■ " •* « 

td their present positions in society or which should hav-e » 

'been- availabl.e to help 1;:hem xeach 'Vheir .g6als • Can parents 

provide" another ^use'ful/perspec tiyc/ Vith their uhique know<^ 

ledge of students ' backgrounds? • Shouid teachers be included 

with their understanding of a c4ntra:i aspect of the students' 

current life? Likewise ; counseling perscY^nel o see* stu- 

delnts in a way "that is mope intimate and Aersonal* They may 



\ 



•have mo're insight in to th c ^ nppr\c; n nrl prnhl^mr. nf nHnrl t?> n i T> 

than either teachers or parents. ^Conuaunityl representatives 
a'nd em^^ar^y^s may be abl^ ;to providt^ useful inf ormat:ion con- 
cerning the societal-relited needis students ^Jiave and . as yet 
are largely unaware of, • Administrators cahprovide a useful 
perspective from'the standpoint Of sGhool-related stu.^ent'^ " ' 
needs. ' Finally, Career development specialists .may be more 
•knowledgeable than any of the " preceding types of individuals, 
in. their understanding of yburig^eople ' s needs and desir^es , . 
given their study and cqmprehensipn 'in- thei field. ^JEach of 
thesQ groups should 'be considerec^, and some decision reached 
as t6 which to include.' Later . or^, some deG^isioh will be * 
called for as" to how each group 's| .input should be weighted. 
But this issue can be considered jhere too. ; If it is felt 
that the opinions of couhsblors and career- development spe- 
cialists should carry the ;nost ^weight , perhaps they, should 
h6 more extensively sampled th^ the other gj^a^ips. Or per- 
haps it i!?ill be- decid ed th at the only truly relevant measure 
should be what studentSj themsfelves saiy ,/and ail other , groups 



/ 
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should be j-gnoreci^. Whatever the decision here, ie is an 



issue" that should be. f aj::ed" and intelligently resolved in 
defining the sampley^^ " . , 

. "•Perhaps a lisef lil- means of illustrat/ing the definition 

■■ • \ • •■■'•/ 
of a sample IS to provide an example of how it might actu- 

• ■ ^* .•■.•/".' • ■ >)• •■ ^ ' 

ally be done, . /The following 4s such an examplc.= It is not 

intended as a 'definition of what should' be done in every .; 

C4se, but merdly illustrates the process involved- and fac- ' 

tors to l:)e- cqns'idered. ' 

' It was -first determined that th^ f m i nwi n g groupc in. - 



the defined- /numbers would be selected: 



/ 



Students — 20% 
Parents--lp% ' 
Teachers-^ 50% 
.Counselors-- 10 0% 
Admin is t r at to r s - - 1 0 0 % 



Actual 'l^Umbers 

7aor 

' 300 
100 
13 
6 



Studfents were selected from a complete alphabeticai* lis 
of ^11 students at the high school, numbered/ and broken dowi 
by g^a^ql -Level; Students were stratified on the following 



crite^ricj: grade point average-, -rank in class, number o'f 
years \at/ the high, ^hool; pa:rent occupation; ;sex/ aptitude, 
and albhabetj/cal breakdown. In tliis v/ay, a fair repreisenta 
^tion waW ina/ured in terms of most of the importan^N^iasing 
f actorsMin /a I-^lgir-^hool. One parent of each of these ^u- 
dent? WciWthon .ch9sen, with minor ad jus tmentsi made to a: 



and an 



J 



the stucltat sample, with an appropriate number'^ randomly 
selec tecl who were,, current ..teachers of . the stude\nts involved , 



(jqaal numbei: chosen who iaad been tej^chers of the 



/, 



scudcnts the previous year/ .Choor.incj 'the cpunselors and 

administrators. Of co.uirsev proved no probiem, wi th a lOOo " 

, • 1 ■ , I ■ ■ I ■ 

representation m each case, 

i . ■ ". . " : - ■ . • ■ 

• A question that should be considered with any sample is 
whAt percent of response is necessary to assure validity. 
Ig 500 . respondents lare chosen, and only 367 actually respond 
to the survey, does one have meaningful results? Unfortun- 
ately., ythere is 'no pat answer to this question. ,Vcirious 
authorities cite figures^ anywhere from 50 to 100 percent as 
♦acceptable levels of response. The best means of determining 



percentage 
of res ponse 



adequacy ii;i this regard is -to scan the demographic data from 
those who have participated," compare it with those who have " 
not, and see whether any patterns are apparent. If parti- 

• ■ .... ^, . , . . - ■ 

cular subgroups have bken included and others -Xeft out, 

probably the data is suspect. If there a^re no such ^patterns 

V ... ' • ^ ■ ■ ■ ■ ' ' - ■ 

it iip probably valid* In any case; it is be$t^to strive for: 

• • , . ' ■ • ■ . ■ ' , ' ' ' J. ^ 

as full a response level as^ pos-sible,,. and to adminl,ster£ the 

asseS/sment in such a way that' respondents axe Imotivated to 

participate.^ Participants should^be followed up if they fail 

to respond -after one contact. , 



- ^ PROGRESS CHECI 

You 'have: decided to conduct a desired diitcomes assessment in 
your school district. The district, con tains ^iye junior high schools, 
(girades 7-9) and three high schools (grades • lO-l^. • The Schools are 
all comparable in size and student makeup. Total district population 

terms of socio-economic status, bbing middle class witii^ypical annual 
average incomes.. You ^ ate' working with limited funds, andNQUst assure 
eq'iii^l representation in terms of sex and grade level... It has been 
determined^ that students should be the. primary ' responcients , but four 
adult groups should also be lightly sampled: counselors , teachers , 
administrators, and parpnt^. You -are "working with district-wide alpha- 
betized lists of students, broken down by grade level. 
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• • 


Determine the following: 








.■ ' 1 




.What percent of each of the groups 'will you^ sample? 








•••• . 


2: 


How will you'^actually pick students from ixhe lists 
provided? . 




... ^ . \' 


t 




3. 


Flow will you choose the adult populations involved? 










4. 


llow will you assure equcil representation by sex and 
grade level? ' ' . 




• . • • V 








What will you regard as an acceptable response lev'el 








6. 


How will you ^chegk . the validity of your data' if the 
level is. below .100%? 












response 


I 




1 Guidelines and examples for 'checking your answers can be 
found In. the preceding pages. > ' , 








DEVELOPING THE INSTRUMENTS. 




Step 
four 


- » 




^ You 


may take a number of courses in obtaining the 










necessary instruments for a desired, outcomes assessriiejit. - 








■r . 


You may 


be abl6 to simply a[dopt an existent s.et of cards.. 




• 






el;Winating<-the' need for any development, work. More likely 










you will 
standard 


need to do si^^mq adapting of 'cards, altering a , 
set of statements, to fit the particular ne^ds^of 




necessary 
instruments 






students 


in yoiir school or district. Or it may be prefer- 










able to develop the cards from scratch, permitting the kind ' 










of structure and grouping mo^st desirable at the site in 










question 


V^hatever course is taken, certain considerations 




. .f • 






must be made and a definite set of stejis should be followed. 










The 


first such consideration ■ involves the question of 










who will 


evaluate the quality of .available cards , and adapt 

■ ■ • .. ^ 
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thein if necessary , or develop now ones if'Lhat course is 
taken, 'If very exLensive adapta*tion of existing cards or ^■ 
.' develop.nte.nt of new one-s is. required, this is a si'zeab,le tasK, 
.and -is usually best accomplished, if assigned to someone* v^^ith 
thQ .necessary time and resources, "Thi^s may mean selecting 
an individual cdunsel^r or teacher with. appropriate know- 
ledge, or hiring someone from within or outside the" school 
with like qualifications. Adequate financial 'and time 
resources must, be, allocated to 'assure a quality product, orJe 
that h^s t^dergonc field testing and necessary revision. 
• ' A card-sort student ^nceds assessment instrument has bo^n 
developed. . The first pha'se of this development was . under- 
taken several years ago. In this phase, a nationwide survey 
and search was conducted to' find, the bdlindairies of a compre- 
hensive guidance system, and to find bases ^for organizing 
the broad variety o^ student- guidance needs. Using a sur- 
,Vey of the literature and content analyses" of available 
.guidcg^ce programs, this phase led to the idcntificatiori of 
the guidance needs of youth across the country. These needs 
wer.e categorized into, six student-centered, areas which 
should be addressed by any comprehensive guidance system. 
These six are: 

1. Vocational planning needs\ 

2. Educational planning needs 

3. Learning-how^to-learn needs \ ■> ' 

4. Personal and social development needs 

5. Social responsibility needs 
•6. Leisure time use needs ' 

' ■ ' '54 . 
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card sort' 
assessment' 
exis ^ent 
cards 
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'• It can be seen tliat ''vocational planning," ncqds consti- 
tute only part of the emphasis, in which *'cairoer" is vir- 
tually synonymous With "life." "Educational planning." needs 
relate to. thQ dirc^ictions a st^lident v;ishes^^^) pursue in edu- 
• cation apart from the educational requirements' for his 
chosen vocational field, "Learning-how- to-learn" nqeds , on 
:the other hand', refer to skills ^which enable an individual • 
'tO; acquire knovrledge or, in essence, learn. These include . 
the skills of reading, listening , iiotetaking ; and more; " 
"Social responsibility ". needs are broader than the social j 
ne'^eds referred to in the personal-social area. The former • 
are needs concerned with. bein^ an effective citizenand 
member of sdciety^, while the latter refer to needs relate 
to interacting with persons in small groups* "Leisure" rVeeds 
refer to t;he desire to struGturej,or plan*' the use oJ free/ time 

Field test^ of this strategy have emphasized assessment 
of youth, needs Hn four of the six life areas. Asspssmeiiit 



materials ^in the other two areas (social responsibiiity ' and 

leisure )j are currently under development* For each area, a 

list of possible need statements was developed. Each need • 

statement,' written on a separate, card, disscribes a type of 

i ■ *''.■■''' 
personal functioning: fpr which a given youth might feel a 

heed to strive;. A decki of cards for each, life area was. 

developed for youth and adulf reactions . This card format 

allov/s respondents to consider and make .decisions about each 

individual statement rather than dealing with all statements 

at once. The areas surveyed, and example need statements 

'f.rom each, ' follow. 



Effective Student Learning Neecl^ ■ 

•■ * . * ■ . ' ' ■' - * ' • * ■ ■ 

i need to. read fcistGir. 

;I need to be more comfortable when giving information or. 
'speaking in class. ^ 

I need to understand hov; I ,am progressing in each class. 
I need to be less nervous when taking exams or tests. 



Educa:^iohal-Vocational Planning- Needs (included arc those " 
educational planning ne.eds which relate directly to 
vocational planning). • . 

I need to knov,'^ what various jobs are like. ^ 

I need to know v/hat -I. can do nov; to prepare for work that 
I want to dc| in^ the future. 

t. ' ■ . _ . ■ _ * 

I need to develop plans which v^;ill help me reach my educa- 
•tional and occupational goals. 

■ i need to. know how to perform v;ell on job applications and 
in interviews. , . 



Since so jaVany need statements v/ere g.^nerated for the 
Persopal-^'^^Social life area, this, area was brpken down into 
tWQ sub-areas: intrapersbnal and interpersonal.'*- 

Intra'perso n al Needs 

I need t6 get in touch with my feelings and understand how • ' 
feelings affect my behavior, 

I need to ''let: go" more— ^^:o have more ,fun. * - 

I need a more constructive way to express anger.. 

- . 1- • ■ .. ■ . ■■ *■ • • ■ 

I need to increase my ability to keep my promises. - - 

Interpersonal Needs 

I need to ' know the things about mc that "buy" others. 

1 need to be a more sharing and trusting person with others. '\ 

I need- to speak up for myself more — to be more aSjSertive when | 
the occasion "demands. ' I * 

I need to better solve problems I. have with my parents. 



.For the purpose of obtaining addrt reactions, each 
statement was rephrased so that it beAan: "Students 
need..," The available list of need' statements is 
"expandable" that ybutli and adults may write their 
•unique needs on^ blank cards' and add therti to those prc- 
sentied, . i - 

As opposed tp using existent decks Af cards the pro- 
cess of generating original- needs statements can bere- 
v/arding and' useful to those planning' a derived outcomes - ■ 
assessment,^ It can give the planner or ^f;otinselor a fuller 
•grasp of tl^4 needs students have, and forcd him to spend 
time gaining an understanding of the full r)^nge of such 
.needs^v^ Arid the process of generating, class^^if ying , , and 
grouping these needs will' probably aid in'hj^ conceptual- 
izations and make him more responsive to- stud^ent needs. * 
Thete are a number of factors to keep in mind \ if this 
coujse is taken. A number of se^mple cards arc^^ usually 
written arid then evaluated to make sure they a^e appro- 
priitq and functional. To be useful as guides |:o program - 
xmproveme:nt, feach statement must reflect a need \eo change 
a bc^havioi:, an a1;titude, or a level of information. The 
fulfillment of a need must be a conditiom that is observ- 
able anJ measurable;^ Vague concepts of psychDlogVLcal 
needs (such as "emotional security " ) will h6t proiide much 
direction to program planners.. They shbulcl be avoided in 
writing need statements.. Need statements must be nleaning- 
ful to students. . Language and Wording should be appropri- 
ate to the age or grade level of student involved. \a 
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statement which reads, for example, "I need to drticulatc 
.grxevances xn a less socially reprehensible manner" prob- 
ably wouldn't have the meaning to a seventh grader th^n 
one which reads "I need • to better control my temper and 
language" v;au Id. Likewise^ statements ' should' reflect rea 
needs of the youth involved. Ignoring such problems as 
drugs and cigarettes, for example, leaves gaps in areas 
meaningful to students. "Statemen't:s should be ones to 
which students would be \yilling to respond, and should 
encourage response. If card statements? suggest the need 
for rationalizations on th-e part of , studeaits , or imply 
guxlt. in the way they are worded,., they . will evoke a. nega- 
tive response to the whole assessment. Students sh6uld 
not feel threatened by the statements. It. should be made 
clear Inhere are no right or wreng' answers , and statements 
should* reflect this orientation. Still another considera- 
tion is that of including need statements which reflect - 
critical requirements for sdccess in the aspect of life 
represented. This means zeroing in on significant factors 
that have been shown to be Important in' the pasti. If 
^xperxence has shown thei-e to five critical factors in 

young person' s development . of personal) and social skills 
these should- be included prominently in the list of need 
statements for this area. ^" 

In addition to writing the actual items, you will nee 
f:o draft ^ome directions to go with them. These direc- 
tions will obviously vary w^-th the' type of instrument you 



have chosen. Soine general\ guiddlinqs apply in almost all 

^ ■ 

cases, however.. First, the directions should be as brief 
and to tHe point as possible..' They should explain the rea- 
son for the instrument/ and what it is attempting to do. 
They should tell th6 respondent ^'precisely what he- i^s sup-- 
posed to do. Be' very concrete^iji. this regard. Witli ques- 
tionnaires, if items are 4:o. be' checked , say so; .if circled, 
indicate thatr/if written out; indicate the length a^d. form 
(sentence, phrase, list) of response you want. Often it i-s 

' ■ V • . ^ 

useful to include a Scunple ifem or i:w o , vjI Lh Lho correc^T 



form of I response- clearly demonstrated. If there are time 
limitations or other pertinent "conditions which. apply £0 
administration of the ins trument , be sure to mention these 
so that respondents can take them into account. Directions 
for the card sort prOcedui'e aire included v;ith the data col- 
lection instrument on pages 63 and 64, and in the Appendix, 
with the orientation materials for. students and instructions 
for interviewers. Well written plirectiaris can help "consi- 
derably in producing consistent jind valid data. 

Once a fair sample of statements has been . generated , 
they should be pilot tested. They can. be tried but on fel-^ 
low counselors , and v/ilT often receive valuable impravement 
^in the process. Along with this, and* more, important , they 
should be • tried on a. few stiudents . comparable to those to be 
included in the .ass'cssmen t . Suggested, improvements sliould 
be gathered directly from the students, .and from observing 
how the process goes and what the results are. Suggested • 
changes can then be made. g/| \ ' 



• ■ ■ • . . .'^ .1 

With this much feedback, the person in charge of gen- 

. ^ . \ . . * ■ . 

erating the cards can; usual^l^^^^ on- an\^ proceed to devq^lop 

the entire range of cards necessary f or \^he assessment, - ^ 

■ - • ^ ■ .... ; . \ 

This includes generating cards in the numbers appropriate 

.for each" of the major are^s being measured\^ It may also 

. . ' ' - . . ■ ■ ' . \ 

r.involve' deciding how cards will, be grouped, i\^ this has not 

been predetermined, • If the numbers of needs g^n'erated don't 

balance across the areas it may be necessary to ^reduce the 

number of cards in one*"^area^ or regroup the cards \into more 

areas, " If such a grouping "process^ is -used, it is i^iportant 

to keep the groups meaningful to Jtudents, and keep i;hem of.. 

a manageable size, "^^ In the latter regard, one will also. 

■need to strike, a bala^nce between comprehensiveness cf t^he 

feards ^nd ease of administrajtion df the assessment. Fie\d 

tests* to date have indicated that more than a total of 1^,0 

cards or more than 30 cards in one group iDecomes unv/ieldy, \ 

Experience also suggests that the typical problem is one of 

' keeping numbers manageable and still being comprehensive, 

.Once the -complete set bf cards has been written aind 

grouped,' they should be field ''-tested , along with dlX other 

instruments ^nd procedures developed. Revisions, suggested 

by the field test shoJuld then be. made', ^ The. cards should be 

in a full-scale assessment only after such a process has 

assured their qua^rity and usefulness. 

The next step in the process of developing instruments 

is that of generating a. data collection instrument. How • 
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will the respdjifeeis of.-students and adults surveyed bo 
recorded? How- v/i|Ll - these recorded respoiVseS; be . summar- . 

ized?' Answers toj-these questions, v/ill depend on- the devel- 

, ■ . n ■ 

opment of an inst]:ument on which to collect the data and 
from which it may be prepared for the computer. 

-This instrumcmt must do a. number of things. In as • . 
much ^^'as the purpo£;e of the card-sort procedure is to, deter- 
.mi-ne a list, of the high ranking - needs of students, the d^ta 
collection instriin^ent should provide a place for the 
five choices among the cards in each graup/ for-i^ample. It 
should also provide' a place for the^ highest choices across • 



all groups to be 1 
priority can -be de 
be- able. to add sta 



is'ted, so that tlySse , areas of highest 



/ 

bermined. It is idesirable for students to 
ements to -those provided, in. case impor- 



tant needs are mis£;ed in compiling the dS'cks , and a place, 
^hould be. provided to record these; , -It is also desirable 

— : — • ■ ' *) ' ■ ■ - • ' , 

for participants to be able to add additional categories 
(areas) to those prbvided, in case important needs are missed 
in the grouping or ^ategorization itself. The opportunity 
for pa'rticipants to contribute open-ended comments, any 
thoughts th^y may h^vre on the assessment *proce<^ure it;s^lf/ 
d reasons why ceriJain . needs' were^im'jDort^ orNunimportant* 



■an 



should be prov^'ided. i Finally, thd format .for this \nstrument 
must .be- S-uoh tha^t it lends -its elf to easy keypunch:i.ng- pind 



summarization in- the next step O'f the assessment. 

• On the next tv;o .pages is an example of such a. data col- 
lection ^instrument,. One developed* for the (field tests of the 
caifd-sort technique. ; The instrument serves one additional \ 
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•fjanction beyond thos'e listed ixji the above paragraph. That. 

: ■ ■. 1 

is that it allows participants ^to rank the. cards two ways, 

..-* ■]. ^ ' ■ <■ 

according to their * needs and according to their wants. In 

this'way, students can indicate discrepancies between wh-at 

th&ir most important needs. are> and with which of those .they 

would<? like the school to help* This^ assumes' there may be 

■ ^ ' ' - . . ' ' ' . ■ ^ 

some needs that students* would ratheri work out for them- ' 

shelves , . making it inappropriate to ' structure guidance and 

counseling programs araund them, . Thus .the important cate- 

gory f Qir_ the program planner becomes ; the . hijgh ^rank±n^^'h^Lan^ 

of students, * . ' 



DA^A COLLECT. ] . )N INSTRUMlilNT 

J' *7! 0 ; ^ " 

• ' . ' i * ■ " 

Thei- information you provide today v;ill be anonymous — ryou do 
not have to sign your name to "this form. Thcr/i* is only one' 
piece of biographical informal:ion which v/ill Tielp us under-, 
stand the groups, of students \/])o par ticipate in this study.- 
Please answer the follpwing ques^tions by placing a chebkmark 
in the appropriate space. i ; " 



'Grade': 



Biographical Information 
Ninth-Tenth' 2 . Ele-venth-Tweif th. 



. DATA SWY.YJ^ . 

. (In the appropriate place, write the nuiriber of the statement 
the- student, seiected.-) . ■ . . . - 



1. ^'NEEDS ■ . .-. ^ 
- . y 

GREEN 

Academic Learning 



1st 2ri'd 3rd . 4th 5th 

Choice" Choice Choice Choici^ Choice 



BLUE ^ • ' 

Educatic^n^l and Vocational^ 

. « ■ - ^ 

ORANGE , - 
Interpersonal* 



, Y^ELLOW • . 

Intrapersonai 



2: WANTS ASSISTANCE 

- — - 

Academic learning 



1st 2nd 



3td 



4th 5th^ 



Choice Choice Choice Choice Choice ' 



■ BLUE 

Educational and Vocjational 

ORANGE ' 
. Interpersonal 

• .'1 ■ , • ' 

YELLOW 

Intrapersonal 



"3. ; "Order 0f"^Xop^ighT^cTioic93 for assis.tanrt:e : 



1st . 2'nd 



3rd 



4i;.lV ^ ^ 5.th 



eth 



7th 



8 th 



Choice Choice Ch oice. Cl:r6tLce Choice Choice Choice Cho-ice 

/I ' ' ■ — -^^—^^—^^ 
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GI^DE; 1.. Ninth-Tenth 



2. 



Eleventh-Twelfth 



4. Write belov/ any "wants*' v/hich you selected in your top: 
20 "Ghdicois fo-r Assistance" . that were not covered in .the 
four -decks. (Those which you wrote out) . 

' -Academic Learnings Needs (Gr^en) . 



Educational and Vocational Needs (Blue) 




Intorpeirsona.l Need3 (Orange) 

. • " * ' , • .. ^ . . . ■ ■ • . - 


Intrapersjonal N^eds. (Yellov;) 


— ^ ^ 



5. Write below any other areas^ or categories of needs v/hich" 
are irnport^ant to you that v;ere not covered by the four 
decks of cards used. Try to be specific and clear .in 
your responses. . .' _ 

^ . • ■ . " . ' X ■ • ■ . 



6. V^rite. below any general reactions or coniiT\ents aboyit this 
interview ^pr. the needs ass-e$-sment procedures . / // 



7. "Write bdlow any additional information you wish to volun- 
teer fGg'jarcling the heeds you selected or your reasons for 



sel^ectincj theta. 

\ 



8". "Write below any comments or sugges'Crtbns-. about c^iirrent .arid 
possible future counseling and gtiidance, protjrcyiiis and 
services in ■ your high. School. ' y./ 



In conjunction wi th developiiiont of tiie id^ita coilection 

:v. ' ^ 

instrumGnt , -y.ou will ' need to obtain access to. a Go/mputGr. 
This involves eitho:^/ using an adaptable .compwter program or 
'developing 'a/^P^^iaily tailored one which v/ill sferve the 
purpose of summarizing th^. data, , and providing the -desired ' 
list of high ranking needs or- wants. Cons'ultation with a 
Computer programmer is a/necessiity in thds phjkse'r 

In.; this, consultatioh^ you wall need to be ablq\ to tell 
the computer pragramriier -/in just what form ypu wish to have 
the data. If you use 3/0 cards^per deck , . for .example, will 
you simply want the statements ranked- from/ one to thirty, 
m te;ms of the. frequency with which each/was chosen? Or ^ 
.will you v/aint data on/ which statements wqre chosen as most 
impor4ant most often^ A third alternative is the weighted 



mean 



a figure which 



by assigning a value 



indicates top ranki/ng needs or wants 
to a st-atement depending .upon the order 



in which it , was picked and the number qif individuals v;ho 
picked it. . Also, youlwill need to. deciide how the =r aw data 
shouldl be. grouped for \anaiysis. VJill/you \^;ant separate * 
figures "fojP'iDoys and gArls? For indil^idual schools within 
the di|Strict? For individual adult ^ubpopalations sampled: 
counsellors, tcjachers, ■ adiiyinistrators/ parents,, cpmmunity ; 
representatives , • and so on? For v'ar/ioiis ages of students.: 
j unior And senior high school .stiideiits , or students in -tl 
first arid last years of high school/, for example? -,For stu- 
dents; |rpm vari^ou^ socio-ecoYiomic l^ackgrourid,s? for stu- 
dents in 1 academic and non-acddemic^ prog^ Each' of th^se 
questi^ona v/ill require - precis^ ansv^ers, The^ computer is a 
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computer 
need's ' 



cbmplcx machino ; ■ 1 1 can prosent: information in a myriad of 

ays.. But it must be . progranuned joroperly first, dwd the 
c6mputer prograinmer can dd this most satisfactorily if ' you 

have first determined, "exactly v;hat information will : be of- 

• ■■ *, ' . . . ■ . ' I • ■ 

• • . . . •• ' . t- ■ ' . . : 

most. use to you. ■ : . / 

Thus far the discussion has revolved mostly aro,und use 
of the card-sort method, one whose advantages have bfeen 
explained.- Space does not permit detailed information on 
all methpdsV but a btief look at- the questionnaire technique 
may prove useful. The three steps in ' the assessment that 
were presented earlier for the card-sort technique are vir- 
tually identical for the questionnaire method (or f^or any 
other- method, for that matter), but v/ith.. development of the 
instruments, obvious differences appear. 

A number of questionnaire foT-maJ;s^fe possible. . One is 
the open-ended written r.esponse,, in which -the respondent 
tells in his ov/n v/ords the- answer to. a question, structures 
his response in the- manner which is ^most meaningful to him, 
•and gives individual insights.. This format usually adds ' to 
the difficulty of summarizing the information, but does 
allow for individual expression.- A second f ormat . is . the 

yes-no questionnaire, in v;hich ' responde"nts simply ag"ree or "~ 

■ - ' ■ ■ ' . . ■ 

disagree with statements which .express opinions on various 

. ■ ■ • ' " . ■ , • ■ / ■ * ■ . ■ ■ , ' 

aspects of the survey 's si^bject . A third example is - tliat 

in which ro£5pondents .are provided multiple choice items, and 

pick the ones that come closest to their t view. *^ This allows 

for greate-r shadings of opinion than the simpld yes-nq ' 



mctl^od, but still providGS somp structure. A fou-rth forma t 
is t|Tat of ranking, in which various ncGds arc lis ted> ' and 
rcspc^ndents are asked' to put iihem in' the order d^^^'^thT^ 
importance. This method is similar tO; the card^sort, but 
is -usually less extensive anjd lacks the individual atten-- 
tion and , provision for added individual responses. A fifth 
format is oratincj> • in v/hich; respondpnts express on a scale 



their feeling about the variables. 



construction of card 



Many of the,' rules: that govern 
sort items apply to questionnaire i'tems." They . should be 
v>^ritten . in a language that is\ completely understandable to 
the intended resf)ondents . Jargon and professional termi- 
nology should be avoided. They should not threaten respon*- 
dents, nor be leading and. suggestive of a proper response. 
Each item- should deal with just one idea or variable. 
Groupings and orderings should^ be meaningful and S(?nsibld. 
A balance should "be strucksbetween comprehensiveness and 
ease of administration, so that respondents ' attention span 
is not exceeded,, leading to biased resjults. Questionnaires 
sho.uld -be pilot and field tested to assure quality. 

PROGRESS CHECK 



Look over the following sample statements, intended for a card- 
sort deck on intrapersonal needs of junior and senior high 
school students. Each represents one or jiiore common weakness 
in s-uch statements. Determine for .each what *^he v;cakness (es) 
• is/are, ci^id briefly detail this after the statements. - 

A. I need to confess that. I st9lQ some money recently. 

B. . I need to self --actualize in" the affective realm more "often. 



C. I need to fight with my classmates less often. 



D. I need to shov/ more affection for my stuffed animal 
collection. * 



E. 'I need to be abie to identify how many weaknesses and^ 
faults I have. . " • 



• F. I need to eat better, and to get more exercise-. 

2. You have .determined that , students are co'ncerned about drugs. 
You must be sure that this concern is repr.esented in a 'list 
of needs statements for the intrapersonal area. Write two ' 
needs statements that satisfy all the criteria outlined in 

^^^^_Jthe text, and ^that cover the area • of concern about drugs . 

A. .,..'.„•..■■■■ . ■ 



B., 



(Since the appij-opriate answers 'are not found on any specific pages, 
check -the Answer Key for acceptable responses.) . . ^ 



ANSWERS . ' " - 



\s5nap BuT>{i?:} uo put?q.s i Dj:oqM mou^ poou i *U 
•^5nap JO sq-oog: JO oii:^; :^noqi? ojoui mou:{ 6z\. poou i *' 

\ ^ ' ^ . : oq qi[5Tiu spuo 

poo& JO soxduiuxp OMj,> •aoTjjns osanoo jo pxnoM s:^uduid:^t?:^s Aul^w *z 



' ^. .•:^uaiuo:^e:^s guo ut spoou om:^ sjdac)o stv[j, • j 

•s:^uepn:^s 04 BuTua:;B3jq:^ puis aAT:^e5au st stlkl 
•X^'T^T^^ s , i spoou , s:^uapn:^s :^som uiojj pohouiea st stii.l . 
•"GOje 5uoJM oif^ UT s,iT ilpoou x^^o? Jod-To:^ UT ue ST STiii •z) 
•XebTuqooif 00:; ST A5pxouTiuJo:^ "a^q:^ ' ssoxButugoiu 'liuToq sopTSoo 
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GROUP ICTIVITY 




r 



.Me.et- v/ith' -several 

■ . ■ • ■ 

o£ your fellow planners/. . 

counselors for. an hour or 

two. . Use -the time for a. 

; brain storming session. 

Explain theft you want to 

. produce a listing' of all 

the ma-jor ne.eds your si^u- 

dents hav^.., Spend roughly 

, 10 or 15 minutes quickly 

listing ■ every type of 

need the group can. think of. Then arrange and. systematize' these in 

.'the- way that seems most- sensible. Next break the. group into one and 

two' man' task forces (depending on the number'* of people availa-ble arid 

* the need, groupings developed) to work on filling out and refining 

each of the areas* of needs. Let the^ask forces work for as long 

as seems productive. Bring everyone back- together for a surhmary 

as the time runs out and people becpme tired. Have each task force 

report briefly on its ' progress Be sure to document all needs and 

ideas that are generated.. This should 'give you a good ^tart on 

■developing items for an ' instrument , or if you decide to use .an 

• - . 

already existent one,, on which one to choose^. It should also, further 
your conceptualization and thinking regarding the needs of students 
in your school or district, and help njiembers oi your staff uniderstand 
'each other better. ... 
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ADMimSTERING THE INSTRUMENTS 

• A numbgr of tasks must be performed to accomplish 
actual administration of the assessment/ once the instru- 
ments are developed. .Interviewers must be selected arid , 
trained./ Respondents must be oriented to the assessment 
process. • Detailed "^'chedales for actual interviews must be 
Worked out and^aJLl-^partXes-^^nfor^Q d of ^^4^^%^^— And the^ 
interviews must be conducted in a way that allov/s flexi- 4 
bilitYf. so that problems may be corrected- and satisfactory 

results.be ..achieved^ Following, are some suggestions on 
■ . . - 

each of these points. - .\ . 

Interviewers may be drawn from a number of soured 

Counselors themselves cannot usually handle the entire 

load.' Outside help must often be sought. Graduate students 

are one ideal source for such help, M the district is ■ 

located near a university community . Undergraduates are 

another such source, providing they are reasonably mature * 

an'd intelligent. Substitute teachers can . serve well in the 

role of interviewer, and are usually familiar with the 

school and how it functions. Para'pfotf essionals v/ho are 

already working in the School may be available. Community 

residents are .. still another source of interviewers: Finally, 

teachers themselves may be able to help, depending on_how 

inter:vieiN(s .are scheduled and whether they -will bo free. 

School personnel may ease the processof assessment through 

• ^ \ . ■ . - ■, " 

their knowledge of normal . school schedules but. .will allow 

. ■ ' ' * ■ \ . ■ * " - ' A ' ' 

students less Reeling of arjonymity ijf gi^Mng .responses.* The 

\ •• • . . ■.^ • . 7 , , -/■ . ' ■ 



number of interviewers needed will depend on the si^e of* 
* • . * . • .. . • ■ • 

the sample*, the number 'pf respondents per interview^ and the 

interview schedule. It is usually best to keep' the schedule 

fairly cbmpact, conducting all interviews within perhaps. a 

week, so that school disruptions, caused by the assessment 



ers should be made in advance, to allow for inevitable 



don't stretch out too' long. Arrangements to pay interview- 
ers should be made in 
. d-elaiya in processTng"./ . ' 

Once interviev;ers are selected, they must be. trained.' 
.This training has been found'^o be most, effective if it. 
employs a combination of workshops, written material^, and 
practice, simulation exercises. The workshops alTov/ the 
planner to fully acquaint the interviewers with, the card- 
sort process r the relevant information about the school,/ 
and to distribute the written instructions and material'/. 
Once such instructions have- been thoroughly absorbed, prac- 
tice sessions can bq. set up which allow interviewers .to 
test their knowledge, role playing. with otkbr trainees and 
switching places so that everyone gets a chance to practice. 
Usually th^ training can* be accomplished in one half-day 
session, perhaps even in as little as two hours. But the 
more thorough it is, the more efficien|: will be. the assess- 
ment. . ^ . 

Wri'tten instructions play a key i:6Tc 
process. They should explain the process in terviewers v;ill 
go ^through 'ifi complete .detail. . An example of such a. sat of 



in the training 



instructions is presented on pages 124-128^^ ^^^^ ^PP9 

•: • - • ■ ■ / 



nd^x. 



It also- serves to define in detailed and. prQcise . terms just 

. . ' ■ ' ' . . ' * •■« ... 

what. happens in the interviews." Then instructions are v;rit- 
ten far use with the four decks, of* cards mentioned in the 
last • section, and for interviews with six stu/leht respond- 
ents 'per interviewer. A close reading ^of the instructions, 
shouid give the planner a clear picture of, jus.t haw- the* 
card-sort procedure v^;orks in actual administration. 

, ' Not only ^ must interviewers be trained/ but students 
must also be; oriented .to. the assessment ' purposes and pro- 
cedures../ One of the biggest problems that can a^rise in such 
an' effort is the confusion and lack of cooperation that will 
ensue if the respondents are not fully informed of the pur- 
pose for the assessment, v;hat they stand to gain from it,i 
and what is expected of ^them in its admitiis . ration. Orien- 
tation sessions are needed to prepar(^ respgridents- in this ' 
regard. If done well^ they can produce enthusiastic coop- 
,era.tioh, rather than passive acceptance or rebellion. These 
orientation sessions can be conducted by, the interviewers 
themselves, once they are trained and familiar 'with the {pur- 
poses and processes of the' assessment. Half hour orienta- 
tion sessions for class-siz^d groups (approxima;tely * 30 stu- 
dents) have worked well in field tests of the assessment 
procedure. They allow f or'^'questions and a real back dnd 
forth cbmmunication process Information can also be pre- 
sented on the evolving nature of careers and the trends , 
which will influence career planning and development for ' i, 
young people. - As withl:he interviewer / training, these 
• orientation sesjsions hay:e been, found ^|o be most effectiv 
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orienting j 
the 

i n terviewees 



if thoy use 'written material as .well as verbal " presentations • 
Sample orientation materials arc present£id on pa.ges 128 and 
129 of Appendix to givQ the planner an idea of the kind of 
orientation of respondents (students and adults ) that is*, 
useful, — ■ [ ■: . • 

Oltie of the issues that will need ,resolution in training 

, the . interview^ers , and orienting the interviewees is the rium- 

\ . , ,,J ^ . \ 

, her of respondents to* be included in each. interview. Exper- 
ience ha.s shown six to eight to be the practical limit of 
pne interviewer ' s .ability . The program planner may wish to 
reduce the number of respondents,, to allow more individual 

attention and e'ase the burden pn the interviewer." Ideal'ly, 
• • • . 

-the system could be used with one respondent per interviewer 
to allow the card-sort 'assessment ibrocedure the maximum 
possible individual attentioji. As "with other aspect^ of the 
■assessment, a balance-must be achieved between, the theoret- 
icaX ideal. and practical necessity. . 

A process that mu^t accompany the training of the 
interviewers and the orienting of the respondents is that of 
working out a detailed schedule for all aspects of the admin- 
istration of the Assessment. This schedule should include 
the training sessions for the interviewers the orientation 
sessions for the students and adults sampled, and the inter- 
views themselves. Many considerations are rtucessary to 
develop these schedules . First, regular school schedules 
should be disrupted as lilptle afe possible. Also, ^ailable 
rooms and space must be considered. Make sure room* assign- 
ments are appropriate io the size of the groups being 



assessment 
administra^ ^ 

tion 
schGdu les 
aud 
concerns 
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-jintegviewGd, More /than one interview at a time may have to 
take -place in a room* Interviews of parents and c6mn\unxty 
representatives are. usually best Scheduled at night , both 
for orientation sessions and actual interviews In some 
cases, interviewers • may have to go to the homes of adult 
respondents/ yhere problems of transportation or available 
time- exist. -I^he schedule should be set as early in thb 
assessment planning procpss as possible, axid should be made 
available to auLl par^ticipating group^:' students, parents , 

•other adults ^chool administrators, and most important. 



interviewers. {Obviously, cooperation will 'be necessary- 



students- v/h en the times come, and* in returning them to their 



between* administrators' and teachers in actually calling 
students- v/hen the 

regular schedfule. • , - 

' ^ ■ ■■■ 

Another essential task -Tnrvolyed in administering the 

•assessment is that of notifying all parties of times and 

places to appear for orientation sessions and interv,iev;s in 

accordance with the schedule that'^is established*. Students, 

tec^phers , administrators and counselors can be^ kept in , 

touch through the usual school channels, in most cases. 

Announcements can be made to remind respondents in advance 

jO.£_JLLp coming sessions, and schedules distributed to teachers 

.to r-elay details. Informing parents and community repre-- * 

sentatives may require a little more v^;ork. Nev/spapcr arti- 

cles should be considered as a means to alert the . community 

to the upcoming assessment. It is best to send those people 

who will actually be interviewed an advance letter which 

explains /che purpose and p.roceclureg of the. assessment, as 



wcX'l as dqrf inin<^ - thG sani|51incj procedure; tliroucjji v;hich they 
v/ere chosen, to participate. The shorter cmd mort} to the 

point this le tter' is, the bette r. It can; <^nd by listing 

! ' . . . • • 

times and. places for the 'orientation sessions., 'either giving 

thd respondpnts'/a chqioe- in. whioli to attend or assigning 



them to a session-. , . . 

The orientation sessions can then liecome a means of 
distributing the sfchgdules for the actual interviews^ This 
should f^low naturally from the explanations of procedures 
and role playing pract^ice built into the orientation. Stu- 
dents^ anli adults alikei should be' becoming involved with; the 
process. -at this' point, | and' this ■involvement v/ill probably 
help' them to rememloer jthe interyiew time. To assure this, 
it may be useful to 'se|id a * f ol]^ov/-up ^letter reminding 



adults of the interview time^ particularly if there is a '/ 
very broad tirpe gap between the orientation and . the inter- 



lev/. 



In an operation as compiejk as the card-sort n^eds 
assessment ; . there is liJcely t'o be an occasional'^i?iitch and 
need, for adjustment. Thus • sc^iedllles 'are* best kept- somewhat 
.^f>I.0xible , so that people may be moved around at the last 
mi/nute if necessary. Likewise, during the actual time of 
a/lmihis tration (be it one da^v^or tv/o weeks) it l]elps 
immensely to' 'establish an^ opkrations headquarterls , ' whpre 
those- in charge may stay to.'k(pep in touch iwith develobments', 
and where interviewers and tjeacher.s may report difficulties 
^.bhat arise* In thi^ Way coordinated sblutf ions may be . , 
worked out quickly, and thci results of the' assessment be 



kept consistent and of a hign quality, 

Aciniinis taring the ins tJGuments is tho soGond. maj9r phaqo 
of conducting a desired out/comes assessment v;hich will vary, 
considerably depx^nding on the kind assessment chosen. 
Administeriiijig cxuestionnaires , or lining a follov/'-up study, 
will call for a process qu^te different ftom that described 
for the card--sort. Again , no attempt will be, made here to 

■i . ■•■ i ..... ' ' ' • 

-des^cnbe the$e processes completely. But a few thoughts on 
each ma'y--pro\^le us&ful to Ifhe program planner^. -* . 

For the 'planr/er who -decidels to. .'se a* questionnaire to 
sample students and adultis, the first difference is that no 

.intjfe?rviewers Will have to be hired and trained. ^ However , 

' j • ' ^ ' 

this ■ d oesn ' t Eliminate the need for. consistency in'cidminis- 
tration- of thjs ins trument. . Thus V whoever ' in faet 'does 
administer it; will need to be orienjl^d, and if .Several 
people are involved (such as teachers) they should j:re 
trained so tl'iit they all follow the .same procedures. ' .Class 
size, groups will *of ten work well' for writlien insit^ramehts of 
the varieties described in ^he previous sectior^. - This will, 
aid in devising .the ' schedule ^ but such , a' sehedulp [will still^ 
need to be documented* as early, as* possible and ciDinmunicated 
to all involved groups. Lik^ise, respondents ^±11 st'ill 
/leed to be oriented to the purposes, and' proccdurds of the 
assessment, and familiarized with the type of qui^tionnaire 
chosen. Parentsj and community representatives n/ay l}o able 

• i ■ • **'••■'■■■■•--/.'/■" - 

to rill out the instruments at <home , but this mhy contribute 
to poor response rate, and more -consistent results hce like- 



ly ifj c6ns;i stent conditions, exist fo.Jt; the adm:j/ni strati on. 



In any case, respondents will still need to be oriented, as 

* . « . ■ ' ' ', ' ^ 

v/ith the cdrd^sorta - ' , 

' ' • - . ? " . • - 

Follow-up studies are usually conducted entirely through 
the mail, and this 'of course leads to a' number of variations 
from either the card-sort or the ^questionnaire. Since even 
iess^' invplvement is requifed on the part of respondents in a 
follow-up study than in a questionnaire^ instruments must be 

kept short. Most students- and ad'ults removed from a' situa- 

■ ■ / • ■ 

tion retain only marginal interest in contributing to its 
a^^provement. A questionnaire- should be eithpr selected or 
deyel6ped, and the suggestions • included in the last,> section 
in this regard aipply' equally well to the .follow-up study • , 
It is best to send an ^advance letter to respondents 'to' prc^ 
pare them fpr the' questionnaire j( or a t least include a cover 
letter explaining the pui^poses' and procedures of the assess- 
ment and suggesting the importan|::e of acquiirihg the student's 
response. A stamped return envelope! should also be included. 
/Some researchers suggest that the best^.time of'' year, to ' 
administer such £ollow-up studies is aroufid Thanksgiving , 
after thg rush of beginning college is over and before the 
ruslii of Christmas has started. , Students ' ■ names should be 
included on the questionnaire, in case they forget tor put *it . 
, there. It may^ be possible' to print the^ ques-tionnairo items 
on the bac}4^^f a. data 'card, with the return address on the 
front, and^-if so, this 'can spCjed the process of . analyzing the 

data considerat)ly . . Mailing costs can be' kept low if third 
. : ■ , . ■ ^, ^ ■ . ■ * . r- ^ 

class rates aire useql* Mailing lists can be updated if' a 

• •:■ ;] ■■ , ' " • • ■ . . / ■ / . 

request to f p^rward , the questipnnaire to a changed ad^ire.ss is 



lncl,doci-.o. t^^ehvelpp^: Follo.w-^p letters shlid be.^ent 
Who do .not return ,^^hoir questio Jaires .' A ' V 



L-ate is especially - important in a" 



f ollov/-up. 



to individuals 
hi<^i response 
study. 

* i'h-?''''°j!?°^ '"^^^ "Appendix A material, pleise do so 
•at thus poiilt. ■ OuGstiortc -5 ■; T-i -^JT^ F-Li-qfat, ao so 

are dircctiv yoi .^.i ^ • . in Ctho ^Progres Si Chqck ' . 
are dir.^ctly r-elated to information f,ound in A^endix A. 

PROGRESS CHECK , 1 / 

Iv Possible sources 'Pf /interviewers include 



A. • 
B. 



C. 

I 

D. 

■E. 



2. 

3. 

4 



It is ^.suaUy-bek to limit the interviewing/ schedule to a 
— '■ — - .- "^ (p. 72) * . ' • 

Arrangements should be made- early for 'the' 
xnteryiewers.- (p. 72) ■ . ^.-^^"^^ne — ^ 



of • 



A. 



B. 



to 
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' 6, / In indlicating •.noGcls> students 
V. ' / level. of help is possible? 



A. The best possible 

B. 0 Beliter than at present 

C. Whalt^could be reasonably 

D. Only partial help- 



Student^ should all' work on'tjhG sam^ color deck at once. 
T 



sliould bo told tp^asisumo, whai.: 
p. 128) 



expected 



(p. 125') 



8.^ 



When students ask .questions during the^ card-sorf, . it is best. to 
provide e^xamples .for them* fp. 12 6) 



\Pr 126) 



When recording the cards, yop should:.- 
.'A, Look art all statements J darefu'lly 

B.. Glance! -statements , occasionally ^ to \chec]; your v/or 
^^'"'TJooX 'ci-t the numberi oh/ the back bf cards 



p. ; Ask the^S'tudent to reac/t to the, st-tlteil^ients.'as you/ record 

; the.. numbers 

^ -,- - 

f 10, The .choic^ of need or want statements .from^l all ifouif decks is^ 



..done- with: 



(p. 126) 



A, -/All qards in each, deck 
The t!|op "five .cards in 



each deck 



\C*.:; The top five ^nd bpttom five card.s in ekch 'deck 



11, 



p.; Cards in only gne deck 

Statements added on blank 
by :, (p. 126) i I ' . 



at a time 
cards are wx-itten Ion "the data shecgp 



B. 



:ou 



The student 



C. ■ The ke.ypu.ncK operator/ 

D, Fellow students- 



Their ^ own' 



12..- Emphasize throughout " the/.card~sor't procedure 
s.ho.uld respond 



in ±erms /o£^; (pp. 56 & 128) (keveral places) 



tha.t students 



heeds ' 



A. . 

: B. 1 Their' needs ahd thej needs of students lik 
. C The needs of all St^uden hs * 
D. The needs' of boyS ^ dnd .girls 



6 

13. , Oi-ientation sessions fcjr students- usually worI:_ btsst ini (p.-I^) 



A. Groups of thjree or 



B.i 
C. 
D. 
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four 

! Groups of ten or tYolve 
Class-^sized groups. 
School-wid^ - groups.. ' V 



th'em 
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14. Student orientation sessions usuallY work bes^t if tlioy present" 
{ . "information on: (p.i' 73) * ' . 

A. , The purposes of the " assessment . . 

H. The processes of the assessment - ' • • 

C. Evolving trends in. careers ' - ' ... - , 

. D. All of the above . • 

.15. The a|-)per practical li.'hiit for the size of interview groups is:' 
(P-- 74) / V; : / . ' \ , . ■ ■ . 

' • A. Three to f'our\ : ■ . 

B. - Six to^ eight \ • « " .. ■ •* . 

,. C. Twenty to* thirty- " • . . . 

D. -." Limitless ' . - ■ ■ " - • " . ' 

16. Parents' and conununity representatives are usually best scheduled 

■ . ■ ■ ■ ■ ■ . ' ' '■ ^ ' : 

A* At .lunch time ■ * ■ . - ■ . 

B. At "night' . < ' . ■ * . " 

C. . Earty^ in thb* morning .. ^ ' . .. 

• D.' At' mid-af te'rnooii '. ' ' - ■ ' " - 

* ' - - . " ' . '■ • ' 

17.. An important, cons idera-t ion in designing schedules is: (p. 74) 

' A. The regular s.chool schedule ^ 
, B. AvaijL^able space j 
C. Size of . the interview gi-oups 

All of the above f . . " - 

18.^ Design- a theoretical schedule for interview.ing -100 students 

s61ected randomly from the • 10th- g'rade at one school, using that 
.school' s regular schedule and available space, as guidelines, and^ 
assuming groups of six students each.- -(See Answer Key) i 
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Meet vvith several of your fellov; planners/counselors . ; "Imagine 
^ that they are- to' be interviewers in the -adminis tration' of the card- 
sort to students .(which they -may be), and that you are- responsible 
. for ' training them. They should pl^y the role of sWpticar trainees , 
questioning the process v/herever they can. Yoi\ should provide answers - 
to their questions from the. " Ins tructions to Interviewers*; (Appendi'x), 
and ^your ov/n knowledge, " if you become stymied, stop the i^ole playing 
temporarily' and v/ork out^ satisfactory answers with t;he group. Keep a 
written a'ccourif of all questions and problems that arise. - 

When all questions afe_ a.nswered and problems worked out, have 
the- group members ^cjivide, into -Erairs- and actually, practice' going .through 
the sorting .proces^ , one acting as the intQrvj^ev/or- and the -other as 
ihe , student. - Again*", encourage questions, 'and continue to keep a 
written account of problems and questions. ■ - V 

■ When, you Jiave finished this group- activity you should have 
accomplished several things. The counselors involved will be able 
^ to help Vith training o.ther'"interviewers, and .to help conduct the' 
'. interviewing themselves; (everyone's knowledge of the process wiil be. 
increased; .many .of (the questiahs and problems that might come upwill 
•have been explored and resolved; and you will have a list of issues 
to be , informed -on, and to cover in the real training. . . 
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SUMMARIZING AND ANALYZING THE DATA, / 

' ■ It i3 at this point that. 'earlier' planning concGrning • 

■ ■ . • * . • 

the computer will pay off. The data must be transferred 

■from the Data Collection Fornv to keypunch cards so that they 
can be tsummarized by the computer. This v/ill require the 
services of a keypunch operator, and a computer programmer 
or technician to conduct the aciiual run. Along with this,, 
the individual op en-ende d responses and added statements 
will need to be gatheired, grouped, and' listed or suiTm3ari2;ed . 

; Groupings and listings can take a variety of forms, but 
usually -follow the same .breakdowns used, for the co^nputer (by^ 

-age/grade level/ sex, or adult subgroup, ' for example), as. 

.well as the natural content breakdowns into which. the com- 
ments and thqugh.ts fall. ' ^ ^ 

The next task is the interpretation of the summaries 
-that come back .from the computer. The illustration shown 
on page -87 gives an example of a printout sheet, showing • • 
the information as it was summarized in the field test. The 

'information at the top of the page shows, that the data refers 
to ninth grade ^achers ,. and specif ically to * their responses 
to the Academic Learning deck of cards. The first column on 
the left lists the . statements (cards) by ndmber. The next 
column shows how many times each was chosen as the most im- 
portant by teachers, follov/cd by its accorded rank (the ' 
lower the rank numerically the higher , its • importance) . . Th'e^ 
next four columns show how many times each was chosen second, 
third, fourth, and fifth,, respectively. Following that, the 
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NcGdlcss' to sciy, all those printouts moan thoro is a 

great deal of cata • to be analyzed ; The next task is^thus to 

determine who is best qwialified to do thik analysis'. If 

counsellors • have been ilivolved with the assessment effort 

■from the start, they may. be excited enough to finally have 

some results to v/ork with that they will bp eager to dig 

into the data themselves. If not, may be necessary for 

the planner to choose or hire spmeorie Witfl the time avail- 
1* * ■ 
--able to serve in this roJLe. ,1 0 

Closely ' related .to deciding who will analyz.e the data 

is deciding iiow different sets of data v;ill be -weighted in. 

the analysris. Will the first choices be counted most heav- 

•/ • ' ■ . ■ . • - . 

• ' . ■ ■ ■ . . ■ 

ily, or the. relative percentage? Itow will these figures bd ' 
reconciled where there are differences? Will the weighted 

^ ■ ■ ■ ■ 

mean be' worked in'to the calculations in some v/ay? -Perhaps 
more important than these questions are .those concerning 
how the responses ©f various subgroups will booweighted. 
Will Cjounsel'orp' opinions be counted more heavily than 
teachers', for example? Parents' opinions m.ore than admin- 
istriators V?. How will the responses of students and adult's 
be weighted? Should student opinions be /counted most impor- 
tant, adult opinions relatively little? Still another 
questi9n .is hov/ open-ended responser and added statements 
will be Untegrat(?d with printout^ datra. Rules should be 
dotcrminoi^, probably i-n conjunction v/ith the advisory group, 
tb guide analysis, of the data and to asisure fairness and the 
balance of results desired. 



Oncc'^ these rules have been do tcriuincd • [and documentiGd 
for future refd'rence and assured fairness) /• the' process" of 
analyiSis can begin. If counselors . anb. the ones to conduct 
this, analysis , they .will need to be trained by the program 
planner to interpret the printout sheets" and to follow the 
rules determined for integrating and 'weighting the informa- 
tion. .Planners must also be available to provide technical 
assistance and. resolve difficulties that ari^e during the 
process of data analysis. ' . ' i . • 



PROGRESS CHECK 



Fill lin the ansv;ers to then following 'questions, by selecting 
the cbrrept responses^from .the list of ten statements • that follows, 
For each question'; put the statements, in the correct chronological 
ordei:, listing ^them by number. Each statement is to be used kpnlv 

once. ■ -r^-":^.^^ . " 1 ■ 



1. List three tasks which 'must be caa: 
computer summary: (p. 35) 



2. 



d out related. to the 



A-' 

B. 

« 



List . three tasks whifch must be carried lOut related to. the 
added statements- and,. open*-ended responses : (p. 35 & 86) * 



A. 
B. 

I 

C. 



List two kinds of v/eighting rules that must be developed and 
documented: (p. 88) . .• 



next column gives the wc^htcd mean.: the. relative percent- 
age of tiAes the "need was high within the top five, as 
opposed to fourth or fifth, for example. Rankings accord- 
ing to weighted means follov;. . Finally, the -percentage of , 
times the need was- chosen Vithin the top five is given, ^in^ 
tbe; ranks according to this computation,' Thus need number - 
three is^highest, having ,be.en chosen *^3,8% of ..the . timc^ within 
the top five, and therefore ranMng number 'one. Such print- 
out sheets were produced across all adult groups at the . 
ninth and twelfth grades^ across^ all "studeuts at those 
levels, and also ■ broken down. by adult . subgroups .and by 



schools (seven) for students. In a;ddition, totals were pro- 
duced for each of the' four 'checks of cards, and -Across all 
de'cks.. ' - . ■ - ■ ' . 




List two tasks tho planner must carry out rclatGd to personnel 
xn this phase: (p. 88) .• ' . , ' 



A...' 



B. 



Statements from v/hich to chfoose are • listed below: ' i. 

1. Deciding on respondent subgroup weighting rules .for data 
analysis . >• ■ « , ■ 

2. Conducting the computer run 

3. : Integrating the added statements and open-ended responses' ' 

witli the computer data ' S. " 

4. Training selected personnel for the data analysis 
■5. Interpreting the computer printout, sheets ' 

6. -Producing a' written sunmiarjy of the added statements and ' 
open-ended responses " , T • . 

7.1, , Deciding^on statistical' weighting rules for- the data' 

analysis , ■ . » • j; \ 

8. Selecting appropriate personnel to analyze and summarize 
the data 

9. Keypunching the data Lnto computer cards -v • ' 
.10... .Gathering the . added statements and open-ended' responses 

and deciding on.groupings for them . . ■ ^ ■ ' 



ANSWERS 



\ 

1 



9Q .' 

93 









& 


•V 




.1 


'a 




L - 


•Y 


'Z 


r ■ 






9 


•n 




OT 


•Y 


'Z 


9 


•D 




Z 


•a 




6 


:v 


•T 



RESULTS TO /\ DEFINITIQN OF THEDESIRED OUTCOP^ES 



Integrating the data from all 'thei scaifces an</ summa^^— 

■ ■ ' ■ ■ A ' • ■ • 1 h ^ ■ ' 1 ' ' 

izing it in tei'ms of High jand low. ranking needb a(i: wants 

produces a full-blowh list'ing of these' across all areas. 



ICS items at 



Particularly significant in this listing are t 
the top and bottom ends of the rankings. The i/ormer are 
high priority items around which ti6 - plan a proc^ram. 1 The 



latter ■ should bh i^arefully con;iparcd to present program 



priorities to cons.iNacr ways in which efforts . in these areas 
imay be made more eff^qiont . ^ Then mor^ - efforts will' be . 
.available for high' need, area^. This' process, will be dis- 

cus*o# in^he next fifiodule in this sej*ies. ^ Also u.sef ul - is' ; 

the analysing of v;hi^h areas rank the highest, as well as 

■ ' ' \ ■ ■ : ■ ■ . V ' , ■ ■ ■'• ■ 

the particular items- Within them. Should Academic Learnih/ 
! " ./ . ■ \ ... 

.needs turn up in six o^ the t0p ten rankings, for .e:iample, - 

\ / ' ■ ■ " ^ ' ■ 

this sugges ts something \^impoi-tant about the program dir^Q--: "- 



tion. 



It may be us;^f ul to i:^reak down, the ov^ralL. results^.. ^ j 



_into subsections; xelatiny \to.' particular content areas or'^t 
subgroups withm the sampled population^. f^c^rhaps* c^ondition 
are very dif fG;rent- aH: one >jurlior high .school than another /j. 
•and thqse diff erences^ .which ^wp.u Id. likely be reflected in 
both parent and student data, ■ ^uggtfst some need for indi- 
vidual, -variation within- the district in program . orientation. 
In short, the data should' be used in the most intelligent* . 
way .possible,' and .only a .planner i\i a given locale can 
determine exatet'ly What this means in ^is .instince. " 




^ Pirobably the .most " important task in this 'la.s t phase of 

desired outcomes assessment is plir«ducing a- 'document which 

' ' • " •' V • ■ ■ i ■ "''"'^ . ' 

supraarizes the outcomes of the asse'ssment in a v/ay that i? 

.; . ■ • ■ , . ■ • . ; . i . ■ 

useful for the ■ subsequejit step^ of prpgrjam/ dovqlppment , and 
for., disseminating thd results to others .l^iPlfe planner v;ill 
need to consider the Ijest format fp.r^hes\e purposes, and 
structure the document accopdrihgly . ' He-v/lll need to consi- 
der who can best ai^i^dTrTw^ the docume\it. Ve'fwill have 
to decide% QjV"'^e.por j:ing strategies to. yarioub grQUps (which 



ai^ab-ly . should include at least the various\ subgroups in 
/the sample ^pulatioii) and thel advisory .grou]^/7"Fo • 
general pop^ulace he. will probably ,keep the report short and 



/ concise, briefly touching on t 



ae central findimgs. Also, 



■* the languacje should avoid technical jlirgon, and be readable 



format ailc 
report to 
involved w 
more, de'tai 
j advisory 'g 
• technical 



by any district parent or res idJ'ent.- -For s-tudents a- similar 



structuring is probably most appropriate. Tlie 
teach(^rs', administrci tors' and those'-mprii closely 
ill probably be more effective if ilj: <3oes into 
1. .-And the~ report foi: the counselors and th^ 
roup should be quite detailed.^/ Lt can employ- the 



language and include ' detaile4>consid6 



avoided for the other groups./ , r- *^ 



.A5 a 
the^planne 
.marizes ,t;h 
gone throu 
phase *qf . t 
tions chos 



part. of this document, or perhaps a/long v/ith it, / 
f should produce a writtcrli statenieint which sum- 

: ■■ ' ' ■ ' 

1 whole desired outcomes " assessnionl: p^-ocess ho lias 
jh. This' shodld describe vfhat |iatp.jpencd . in cac 
le ; prdcess the central purposes be.hijnd it.,- 'direc- 



rations- 




s ummai;^! zi 
and 
vapor tin 
r($S| ul ts 



1 




/ 



docunientatic 

' J ■ 

assessment 



-^. ^ r 
' ■ .1 ) / 



• ■ .. • • ■ , :• ^ .; I- ■ ' : ■ ■ ■ ■ 

•fdr. important phases, and. -any other .information ossuhtial- to 

an understanding of the >^ssc$smci;it procerus* • ,lt sihou'ld 

include soViiG background and history, of. the iocalcv and 'the 

• ♦ ■ * • . . • • • • ' 

events th^t led to the assessment J .• iSuch a document becomes 

; invc^luable with, the ""passage of time,, and wi|.l serve .v;cir to 

f ulf il^^^'in^ communication needs V7ith various groups as 

v/ell;. Follov;inc[_is an outline for such a document, one 

which assumes the a-ssessnient was district-wide* 

I. 



•II, 



I 




Th 



Background jind\ Purposes of tl'irr 'Pj^anning Actlviti^e^ 
Planning Actiivities ^t vMl Schoo-ls . I 
A. Design ' ' ' 

•B. Sanipling^ ^ . • ^ _ 

. Instruments ' • ' / • . 

. D/ i ,a . CollArtion Procedures and Schedule 
E. Data Analysis Procedures • / 

PvGsults, Conclusions , *and Implications ' ' < 
(by -school) / / 

A. KSchool Number One — llig/a and 16w needs and 
wants .aa^percoived by /students and adults, 

7 (iricluding. resolution/ of disdrepancies) 
rolatcd to the de sir q'd prog rim... 

B. School Number Two — ; ' i 

C. / Etc. . . ■ ■ . ' ' ^. V 

District-wide Sumnhary, C|onclusions , and .V i . 
RecommGndatiohs ' ^ . . ' * ' \J 

ve cJocidment, and par'Eicrularly the .summary of 



high anA Iq/w' needs and wants Velated to the desired prptfram, ^ 
must b(fe submitted, to the advisory group. With completion of'- 

;/■;.•/; ; ' • \ ■ . ' y- 

this s^tep, the p)lann6r .!is ready to move' on' to' the next phase 

of prp.gr am' improK^eiaent , that of . corApai-ing tlio ' inJJoVmation 

/-.■■■ ' • ^ " • ' • ^ , ' - * 

pro.di/ded by the current status asses*sment and the desired. 

outcjbnjes asses snyent to deterjnine program needs arid changes. 

He i|s ajLso ready to move- on' to'- the 'next packag'e (module) , 

i ■ ' • . ■ ■ ' • •/ . ' . ' - . 

/ 96 
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DGvelopinq Program Needs and DGV^loping Goal Statements 
and Student Performance Objectives , 



GROUP ACTIVITY 




• " Meet v/ith several of your fellow planners/counselors for 
roughly an' hour . irave them pretend they, are members of :your- 
advisory board, and that you are presenting the results of the ^ 

^desired outcomes assessment to them. Explain the 'outline of the 
report and the form the information will be in when it is actually . 
presented. Have j:hem look for holes andytjaps, taking the point of 

^view .of ' actual individuals on the advisory board." Looking back, 
from the perspective ofe -a completed assessment, when you are ^about 
to use the informa'tion that was /produced, can often provoke va.luable 
insights. Keep a wri^tten account of points -brought up, and be sur^- 
to account for them' when . you design your. real-life assessment , 




•.-•■'■SI 

. You have now covered a sdbstantial body of material- which . 
includes: • • ^ ^ • . i. 

(a) All overview of desired . outcome^ assessments , including their 
rationale, the nine dimensions on which approaches y^zf f and five 
practicalt examples of method? that have actually fceen used. 

(b) A derailed account of points to.be considered ■ during, the follow- 
ing seven ...steps of the ' assessment process: 

1. Planning and Cpordinating with advisory Groups 

2. Iderrtif ying .the Areas on Which the Assessment ^ill Focus 
- 3. Defining and Selecting the Sample .( 

4 . Deve loping the Instruments •• ^ ' . 

5. Administering the Instruments * . • ^ 
" 6. Summarizing dnd Analyzi-cig the Data 

7.- Applying thi- Results to a Definition of the Desired Outcomes 
(.c) :Eight Progress Checks and Four Group Activities. . - 

. - ' • * * . ' • . • " i' 

The -Simulation Activity which-^ follows will give you an oppqrr 

tunity >-o apply, the knowledge and skills you have gained .to a theor- 

eticai situqition^/and run through each . of the steps to ^practice your. 

abilities. ■ Next, the Application Procedures will . of/.er. suggestions 



'' ........... 



and guidelines for applying your -knowledge and skills tc* your^ov>f.n ... 
real'-life situation. Following that is a post-test which, will permit 
vou to see whether you have 'achieved all the objectives of. -the module. ^ • 
Finally, oadditional. references are listed in case jou wish to pursue 
your study X)f dpsired'^bwtcpmes assessments further. / • ; 



Sli^ULATION 



The -questions ori the next page relate to the theoretical 
situation jle scribed below. Read over that description- an'd then 
proceed with the exercises that follov;. ' (Maiy i^e done as small 
.group exergise.) ■ • 



You are the 'district administrator. of guidance and counseling 
for tho Lillington School District, a suburban district made up 
of seven elementary schools, four junior ihigh schools, and two 
high schools: Of late you have sensed that the schools are Jiavihg 
several problems. Students' dissatisfaction with the role the school 
is playing in their lives s.aems high, their preparation for the 
world they find on leaving schpol seems weak', and their need :£o 
better adjii*st to" personal -problems has been ^in_ evidence recently. ^ 
In addition, there has seemed '^to be 'a kind of drift in the guidance 

. ■ ' . '^-y-^ - ' ■■ • ■. ... 

and counseling department, and'^you sense a lack of real coordi- 

■ . . ■ o " . 
• ■ • . ! ■ 

uation^ among the efforts of various counselors. The department 



/ 



has no gerierally de|i.ned goals and objectives at this point.. You/ 
are thinking of unddrta'king a desired outcomes assessment,. b<:t have, 
limited . funding available.^ ' .? ,'. 
The district jp hor-wming incroasinglys^iversc . One epd of it 



is istill la>-yely rural, while the other end is rapidly urbanizing. 
The tv/o 'junior liigh schools' (to be referred to ag) .pumbers 1 ahd 2) 
and one high school (number. 3) serving the rural part of the di€.trict 
■ ar.e relatively old and small. Their urban counterparts, . junior high 
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schools numbers 4 and 5- and high school , number 6 , are .newer , l^irger , 
and more: complex. Families iri the^^-^rc urban area ' have, an 'average of 
$3,0'0'0 per year higher" incomes. * . , v 
The- population breakdown for the six schools is (grades 7-9 are 
.junior high school; 10-12 are high school): \. • . 
School: 1 2 3 4 6 



Subgroups : 



Students - 


431 


■ 450 


802 


981. 


1, 


358 


2,193 


Administrators 


1 


1 


2 


2 






4 


Teachers 


19 


18 


38 


■37 




.5 4 


83 


Counselors 


1 


1 


2% 


2 




3 


. 5 


_ Alphabetical 


lists 


of studQpts 


broken 


dovm 


by 


grade 


level 



available for each scliool. Alphab'etized-^tists of administrators, 
teachers, .Aid counselors are also available for each school. ^Lists 
of teachers^are 'bi'oken dov;n by subject area and ^grade level. 



1, - V/rite a definition of the purposes of the desired outcomes 

assessment, c.nd describe, how information that will be gathered 
^ will help in dealing with these. . 



\ 
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2. Choose and justify the type of desired outcomes, assessment 
. • procedure you will. use. •' ' • . . ^ 



S.-: Define the pre^3e f6cus for. the assessment, and suggest ■ 
ways you might^Tind additional information to help you in 
thi^ ta^k. ^ ' . . 



4. • Define the sample for the assessment, including: 
aw The subgroups ypu v;ill . include. 

b. HOW the sample wil^. be divided across schools. 



c. The actual nuntber af respondents JiO be included in each • 
subgroup. ^ § ' 



d.- The process you will, us^ to arrive at* the actual numbers* 

. e. " How you; will attdm^t to assure a representative sample 
(mention all variables allowed for in this regard). 



fVs^* How you will actually . piojc the names. 
g. What you will regard as an acceptable response level. 



h,. How you will check the validity bf your data if the 
• response level is below 100%. • ; 

5. Draft 20' sajiiple itjem^. to fit thQ ^procedujie you chose in 
question. 2.' Inclulie each of ith^ areas o^ Tocus you have 
defined in question 3. Be Sure your sample items are: 

a. Behaviorally measuit'able -variables 

b. Meaningfully >>rtDrded for students . . • 
' c. * Conceicned with important, variables 

^ d. Grouped. appropriately • 

f V Touching on only .one variable each 
g.. Non-leading 

IaIsq draft a brief set*of instructions for use with the 



-i terns i Use non -module ^>cip^r^ gar -.those -ac-tivxtlGS- > and kee p 

your drafts with the module when you have finished. 

5. Pilot test youfV20 i-tem instrument by f inding a group of ' 10 
to^ 15 available/\ind appropriate subjects (students , fellow 
, planners/counselors/ boy/girl scout troop, group of ncighbqr- 
hood youth) and conducting .an actual adminigtratian , with all| 
procedures and conditions, as they would exist in reality. • 
Briefly d*(^3cribe the- process/, and mention any seriou^ problems 
that aroGe'X - " 



Summarize your data in table form, and write a brief narra- 
tive tha^t KigRXights the major "findings"- of your study^ 
• • • 
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SliULATION FEEDBACK 



4 4 



\ 



The desired outcomes assessni(^f?fc^will serve ntaiiy functions. It \ 
will giv-e you precise empirical . .^ata on the needs of the district's 
youth, thereby determining the extent of those problems you have 
sensed, and others you may. not have sensed. It should help to 
unite dounselors around common goals ' scientifically arrived at. 
In le'ading to beh^viorally defined goals and^ objectives it 
should remove feh-a drift fJi^m the departm^^nt and put it on a* well- 
defined course which allows for measureihent of progress and 
sucd^'ss. By having comprehensive, district-wide , ^reliable * 

• . ■ 0 . -. 

information on students' needs relative to the role of school 
in" their lives, their post 'high school worlds, and their personal. 
problems, you can not only improve student attitudes about school, 
but: give them more self confidence about tfieir future;, better 
opportunities for career s-ticccss, and better feelings about, 
themselves.^ Some of theae objectives- will work to improve^ 



academic performances as v;ell. . \ 

'"-^'^- ^ Vf^^T ^;:^y have included other thoughts a nd suggestions in - ■ * 

- - — „„, . ,. :„ , , .; . ^ . < ,„■ ' 

your response. Many good ones are possible. The above summar=~ 

izcs a few. „• . • ; . ^ 

With a district • this large you can' probably eliminate personal 
interviev/s if you hope to include a representative sampj?i3* 
Since you want information from current . stu'donts , the Delphi 
Technique and Follow-Up Study seem out. If you had predefined 
g6als you v/anted to define behaviorally tjie Critical • Incident 
'Technique might workr but you don't. This leaves any one .of 

• . / • . • ' . ■ • ' ■ 104: ■ . 
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ah' as^sortment of' questiorinaircs * or tKe ,car4 sort. One crii;orion 
tp^'consider in <:hbosincj .among* these is the amo\ant of funding. , • 
Since yours is" limited you'will either "need "to choose a** careful 
sampling , procedure or use a, fairly 'brief questionnaire* Another^ 
criterion concerns' how .Comprehensive you would, like your da^ta to 
be. Still another concerns'' the form in which ^^otj/Vould like «it 
to appear. No one choice 'is absolutely correct;; this is 'up to . • 
you. But, be sure you have considered these factors in rtfeking 
your choice, • • * • „ 

Again, no one "correct'' answer exis'ts. - iiut the indications you . 
have siiij.gest that you ne,edVtc5 ^ learn about students' attitudes 
toward schoc^ and the causes for those attitudes, career and . 



vocational education- needs of students; and the kinds of student 
personal -problems most prevalent? Perhaps a survey of counselors 
might help^to detei'mine [additional impori^ant areas to probe ^:nto. 
likewise, teachers may have suggestions to off cr. . -^S'to4e^^^ Igad-, 
ers represent another' good source of information at thiis 
tive *stage. Remember, you're defiling the., areas ' you want to 
learn abput, not yet^^^wlri4::4ng specific items. • • . _ 

These responses will depend on your^^. previous decisions and ^. 
responses //and- your own judgment. They'^provide'p^ct ice toward 
conducting your own real-^life desired outcomes assessment. If 
there are questions'- or problems, go back and ' check the appropriate 
sections of the module. * ' ^ 



APPLICATION I'BOCEDURES 



All res'ponses in this section phould be appropriate . to your \^ 
own setting >• The questions are : c^p^^igned to help yoyi. thirik through- ^ 

the. factors and probrems that should Be; consiVfttred^ so that you can 

.• ■ ' . . ■ . ' • . • 

.. actually conduct. ryour own desdted oatcomes assessment*. . Job. down ^ 

•any responses, examples, or^ guidelines you can^ think of- for e^ach. ^ 

1. " Wh^at arb thfe condiiiiqns that argue for a desired outcomes t 
assessment?. Against it? Review pages 19 through^2 in the' 
Overview st!Gtion if you have forgotten the factors that usi^ally " . 
■ , ' pertain in such considerations . ' Jf ^ 



0 
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2. Who Would pr^bably'b.e oil your 'advisory .boar<3?' How 'might you • 
design the ideal advisory bioard? How will, the makeup" 6f this*^ 
board affect the orientation and scope of the assessment? 
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"f^Jiat personnel, are available. t,b help v;ith such an assessmemt? 
^.Specif ically^ who can help^ wi'thr . * . ' . 

a. Initial 'planning? . . -^--^'^ 

S. ' Def iTiing •and\ seiec.ting .th§'> sample? . • \ ^ . . 

' " • , . ■ • ■■ • 

c. . Writing anstruments,? . > . ' 

• •■ ; \ o * „ . • • . • . ... 

d. Adrtrini^tering the instruments^ : ' 

e. . Keyp.uuqhin<^' 'completer cardsv programitiing the computer and . 

man'ag^ing the computer run, analyzing printout sheets^ 
intgigrating open-ended irespon pfroducing. a data, 

, summary? * " < » . . 



' f, Writing a report v;hich •summarizes the. desired outcomes and 
^ thd jg;^ssessmeiit process? ' . - • - 



g. Disseminating information at various stages? 



What would be key areas on^which to focus an assessinent? Does 
yie present philosophy of your department . provide suggestions? 
Do aliready apparent needs for change suggest directions? How 
do your .particular conditions--district locale , socio-economic 
status, and present trends--bear .pn the issue? 



What assessment procnsdure seems most cippropria te givcii your 
conditions and needs?^ Do the advantages of, this procedure 
argue convincingly for its use?. Are the di^advaintages mini- 
itiized by your situation? . Consider a-ltefftratives . 



Define precisely ^vhat information. would be most helpful in 
providing solutions to the^ problems you face? In wh^t form 
would this information be. nicst uso-ful?^ How extensive should 
it bt?? Consider alternatives— ^are there other kinds of infor 
mation that would. work equally as well? * ' ; • 




Ha,ve there' been any barriers to conducting a desired outcomes 
assessments to date? What a^re they? Have you encountered any 
resistance* to solving thfe problems a desired outcomes assess- 
ment will help with?. Are there ways to overcome these now» 
with the help 'of the as.sessmei^t? [ 
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8.- ' vi/hat kind' of time framfi dd you- "envision for conducting the'. ■ 
' desired outcomos assessment? Wh(^n would you start?. When ^ . 
. wgVicT Wu-complete the planning stagtes? Instrument develop- 
• -ment? Admihistratiort? 'Data summaries? Repprts of ^he find- 

• . ings? What,.are alternative schedules? Which is. most ideal? 
■'.".Most feasible? ' '.' • . ■*.. ■ ;- 



V.' 



Imagine yourself looking" back on an 'assessment' effort^, with . . 
a eorapleted report in. your hand. What, does thp report tell you . 
that you don't "know now,, but would, likex^to .know? , What key infor- 
mation' has been added- so that programs mW/he improved? Be sure 
to. plan for obtaining these types of infpb^nation when actually ^. 
■designing your- "assessment. ^ 



10. How-will you coordinate the desired outcomes asses.sment and, . 
• ' -current' status assessment? "Assure meaningful comparison data 
from the two? Coordinate t|ie time schedules fo.r-the two? 
PrpducG a list, of goals and objectives, for' yo-iir program from 
,a 'study; of the tv;o? Answers to "these qliGstipns and help with 
- — this -task a-ro .provided in the ,ne.xt modu'.e. ■ ' 



POST AS 



« 




' ' If 'you have read the 'instruct-ional material in- this module ' " 
caref^jlly, and completed each af the progress "checks and exerciser, 
then answering the questions in this post test will be largely -a/ • 
matter of reviewing your pr<5vious responses". In any event, it ■ - 
proyi'des a useful final "check on whether you have achieved the ; 
objectives ■ set out at the beginning Of .the module, (p, 7) r" _ ' 



i. Define each of the following in .a sentence: 
•Current .status assessment:. 



\ 



DGsired ■du'tconies assessment: 



Needs assessment: 



.Explain the relationship among, the three in a brief, paragrciph: 



2. 



Write a paragrapfx ■ dcf ining the purpose and : importance of, 
conducting ' a" desired outcom^es assessment . Cpp.; 19--22j 



" DescribG. in . a sentence, each'' the nine dimGns.ions on which 
desi^'ecT outcomcr^- ■ assessment approacKds .vary, as they-' were 
•described in" the module, (pp. 22-2-7) ' ■ . 
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f: 
II. 

I.v 



4. Describe briefly five . alt e relative desired- outcbmQS assessment 
approaches tihat .have been used in. the past , and state one.. / „' 
.advantage- and one disadvantage for each. (pp.. 29-33) y 



Advantage : 
DisadvantageH 

-B. 



Advantage : - ■ 
. ^ D is advantage : ' 



1. 



■ Advantage : 
Disadvantage : 



D. 



Adyantcigc: 
Disadvantage 
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.Advantage: 
,:Disadvantage 



f 



.5. - List the seven ' steps ..cpmrnon to all desired-- outcomes' assessments. 

- . . -. (pp. ^5-36) « 



A. 

B.' 

y- ■ 

D. 
E. 
F. 
G. 



6. /VJrite a 'brief paragraph which ^lescribes the purpose and impor- 
tance of planning and coordinating desired outcomes assessments 
• ■ : \ with^the advisory groug) (s) , including the principal typesy^f 
' . ".activities necessary to accomplish -this step. (pp.. 37-39) 



7 4 List "throe important factors that should be considered in 

identifying the areas on which the desired outcomes assessment 
. will "focus.- (pp. 43-44). . ■ . . ■ - _ 

• A. - • ' ' ' ■ ' .'^ 
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Dief ino : ^ . . . ■ > / * . 

A.* .Three important considerations in selectrng tM sample :, (pp. 48 



/ . 



Q. Two •alternative\sampling strategies.:/ ^ . - 

- ■ ■ / ■ f ■ ■■ •.. • . 

2. . ^ ■ . . ■ ^ ■/ 

C. The procedure most, appropriate to your, situation:; • 

D. The/ rationale for this choice: 

List: .'.'>■ " . " 

Three important considerations, in^.^developing '^the ins't^rnents 
and procedures for a'de&ired outcomes assessment : CPP- 57-61) 

■2. : ' - ■ • ■" ■ - ' / • 



3. 



B; The stejp's you will go through- in thi's development: ■ 
^g. Ten examples of items you have generated for your instrument: 



10. List: ' • . - . . , ■ • 

' ■ • . . ' " . ■' } ' 

A. Three important^considerations in administering the 4nstru- 

," • ment you have developed: (pp. 71-^79) ;| 

'i.'* • ■ • ■ .■' . ' ■ i 

- ■ - 2. ■ . ^ ■ ; ■ : i, ■■■■ ' 

- ■ -3. ... ■ ; ■ . . . ... , ..: . ., ■ 

E. The steps- you will go through ir\ this administration:. 



11. Identity thrfee app'rbprrat^ proced^ds involved in summarizing 
and analyzing the/data y§u ihave collected: (pp. 85789)'. 



A. 



B. 
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12. Summarize throe important coilsideratiori^ in applying the. 
• results of your cUta analysis to . a: "dGf inition. of the desired 
outcomes for^,your. program: (pp,. 9i-94) . r . , 

-'■ •■ • • • • ■ - ' •*'• .. ■ * ■ . . 



C. 



13;. Identify: (pp. 92-94) 



A. Three target aud-iences for your rfeport whidh summarizes 
the procedures and products of your desired outcomes 
assessment: . - ' , . 

' : 2; ■ ■ • / ■ " ■: ■ • ; •' < , ' " 



B. 



The topics of your report: 



!c.„ Ways in wh'dch the report will be° tailored to each of *the 
audience^s listed in part A of this question: 

n 1. .• . • 



2. 



3. 



1 
I 



: V. 
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POST MSESSMEJiT FEEDBACK 

1. '-Define each of the following in a sentence: (l^V 7) ^ 



Cur-rent . s batu^ assessnicpt : The empirical- determination of the 
o'biectivGsV activities, resource allocations^ and qutcomes of a 
program as it no.v' exists. , . . ^ "\ , 

De sired, outcomes assessment : .The -ernpir ical dG]:ermina't:ion . of the 
gap between current status and iddar status -for . either an indi- 



vidual or a program,. 
Needs assessment ?" - The empirical . determination of the gap between 
current status and" ideal status for eithe-r an individual or a 
pragram. ^. . ' ' ' ... ^ . 

Explain the relatidnship among" the threfe in a brief paragraph : 

Current status measures where* things are at present, 
desired outcomes measure where they v/quld be ideally, and needs 
represent the gap betv/een> these two. 



2. Write a parag.raph defining the -purpose and impo tance of con- 
' ■ ductin ' a desired, outcomes assessment: (pp. 19-22) • ^ 



^.^--^ There are numerous reasons to conduct- desired outcomes ' - 
assessmenjts. They supply, the means for accountability in- 
guidance^na-^-e^xuii^seling programs; give CO empiri'cal data ^ 

- around which to strTiCtrur^^^^ograms; identify youth needs rather 
than system- needs ; provideTctJmpriQher^ instead of iso- 

lated guesses; increase program coor^-nat^^^ initiate a - 

process of planning cin'd evaluation designed^~~tcr-4^;eLQP^^ 
abreast of' the present. 



3. 



Describe- in a-sentence each of the nine dimensions ' on which 
desired* outcomes assessment approaches** vary , as they were 

described in the -moduli. (PP* ?2-27) 

■ ' ■■ ■ ■ . - ■ . 

A. Direction and Content " 6T -Focu's —This^ refers to the nature 
. of the content , of. the approach / .whetl>er it has a problem 



or solution focusr; or some other focus. 



Breadth of Focus-^-This refers to how broad ^^^^^^^^''^^^^^ 

whether it "vyiTl. • 
or morcf^ lightly iivto" 



comprohcnsK''c the as ses^sm en't . kiTX' bc, 
delve d eeply into^ one or two ai:t:^as/ 



many. • : • . 

Depth of Coverage — This refers to the overall lengtb of 
of the assessment, - how deeply it probes^ into the areasr 
•it does focus on. 



ERIC 



113 



116 



i^* Oe(?ree o£ Structure — Assessments can ask for hjighly struC- 
tured responses, suclv as checks or rankings, or for'^open 
endefd responses, which allow the respondent to phra^'*fe his 
own response. « ... ^ 

B. P ersonalization — -Some assessments are administered to 

thoifsands iinpersonally , others to a few through personal 
interviews; The degree of personal attentiqn received by. 
. the respondent is the referent. / . 

F. Sampling Considerations — This refers to who will bo included 
as respondents in the administration of the asfees^sjtient r 

-G.^ nnnii^y^i-^ Frvvir factors are particularly impor tant in deter- 
mining the quality oif an assessment.^ They are valldityv i 
3 reliability, generalizabiirty , and replicability ^ . 

H. ^ Availability and Costs — This refers to v/hcther instruments 

and procedures are jcommercially available or must be indi- 
^vidually '^developed, ahd'in either case the expense involved. 

I. o Ease of Administration — Appropriatenoss of the instrument, 

"e^se of data processing / time of administration,; antpunt and 
n'aturc of training, scheduling flexibili.ty, and practical 
usability, all pertain here.' 

Describe briefly five alternative desired outcomes, assessment 
approaches that have been used ^.in -thQ *past, and^st-ate one cjdyaii- 
tage:and one disadvantage for each. , (pp-. 29-^33) ^ - 

A. The Card Sort "*,Asks jre^spondents to rank statemfents \each 
printed on a separate 'ca,rd in terms of their impqrtance. to 
him. ' ' • / , : 

Advantage : * ^ Flexible, f airly .personal/ provides structure 
v/hire~1allowing for individual responses , gives 
. a ranking of needs. 

' Disadvantage: Less generally available than the qucstaonnair^^ 
g ahcl more^^complex to administer, usually using 
■a smaller sample. 

^' • ^ . " - . • ■ • : . 

B. The Questionnaire — Paper and penpil ins truments which have. , 
the individual respond-by -checking, rating,* choosing among - 
several choices, ranking, pr writing ^ii^ his own response. . . 

y'Uivantacjc: Host wi.dcly available and i^sy to . admitiigatej;:^ — 

_ j?covi ts^^li^^ • - * 

Disadvantage: Often provides legs depth of coverage, involves 
participants less , lacks ' pe f sdnal iza^ioh . , 



C. The Follow-Up Study --Usually a questibhnairer' acj^i * 

* to respondents now removed from the program on "which they • 
:_. are su pplying information, usually . ad^^ 
the^mail. .. ■ . ■ „ / • 

" Advantage: Gives benefit of hindsight, instruments easy. ^ . 

- to d^v^lop ^'^^^'^^^ ^^^y '^^^ ^^^'^-^ ^ 

'administration' fairly easy. • - ; .* 

Disadvantage; Imper^sonalizatibn, narrowne of sample, lack 
' * of deep coverage. ^ ^ - . ." 

D. The Critical Incident Technique — A mesons of defi^ing^ goals 
behaviorally through observational, t^ftniques , and then . : ; 
asses^sing the current status and need5;>r6lative to the . 

— hphnvinrally^defined goals. ' : . - 



Advantage:. Uses valuable- dbservational techniques goes 
V intoi depth, inyolve^s participants. - ' . . 

Disadvantage: Reqiiifes. training, extensive work, and us'es 
' ^predefined goals. ^ * > 

"■ ' . ^ • . , • ' ' ■ f ■ - . ■ — •. - . • ^ ..- '-'V- ' ' . 

E. The Delphi Technique — Ameans of sampling experts (usually) 
.. 'and giving them feedback from others which is. usefiU^ . . . 
bringing diverse opinions together aild arriving aF'afe*^^ 
outcomes and goals. 

• Advantage: Prompts deep exploration, elipits fine dis'-tinc- 
tions , -provides good. insights , aids in .ranking . 
- priorities.. . V - ^ 

. DLsadv.antage: Narrow sample, much V7orJ^ in terms of tai>u- 

■ ^ . * ,^ lation, record keeping, • and mailing, and;, 

■ ' provides gongensus , which may be a disadr- 

' V . / vantage i ' 

• " ■> ■ .• ' ■ 

. . ■ , ^ ' * ■ ' ,■ ^ 

5.. Lis:t the seven steps common to all desired outcomes as se^ssmen^.^s. 
(pp. 35-36) > . ' -' /'^^'^^'^'r-. 

A. Planning' and coordinating with the advisory group (s) • 

' • B. Identifying .the areas 9n which the assessment will fociis. 

^ C'.' "Defining and selecting the sample* , . 

Dl DpvSlpping the instruments. \ \ ' j 



e/. Administering the instruments. 

F. Summarizing, and analyzing the^data. . . ^ 

G. Applying the results to a definitibn. of the desired outcomes 

■ " • ■.; ; . . ..'lis' • • ■ 



5/ VJrite a. brief paragraph which describes the- purpose and impor- 
tance of planning and'cpdrdinating desired outcomes assessments 
with the .^advisory group (s) , including the principal types of 
/"activities nece.s'sary^to accop\plish this*stef>, (pp.. 37-39) 

This is inipprtaWt to^ Assure enthusiastic cooperation, .from the 
body which will be able to put the results, of the ass<2ssment to 
use in making real improvements. Planning activities which are, 
key include: . cpnynunicating the purposes and procedures of ■ the 
assessment tp. those to b^^invoived and af'fected; .choosing the ; 
best p-rocedure;for the'as^eS^ment; p'lanning subs^uent steps and 
finding personnel to fill the' nQces^ary roles ; coordinating this' 
~ phase with the current status . assessment.; assuring meaningful 
data; and making necessary considerations for computer use. J 
* *• ' ' * ■' " . ' ■ ' ■ . ' • „ - » 

7. L.ist'. three important factors that * should be considered in iclenti 
fying the areas on which the desired outcomes assessment will 

A* Defining "it's purpose', 

vB. -Considering various procedures ^ and their' usefulness relate" 
. to the ' def ined purposes . 

C. • Establishing decision processes and rules, and c|ocumenting 
\ these . ' . ■ > ^ 



Define: (pp. 48-50) . - ' 

2V. Three important considerations in selecting the sample: .. 

1. Assuring a representative sample. ... - • 

2. . Minimizing . sampling .error by includ,ing jadequate numbers. 

3 . " Selection of 'Subgroups to bc'MnclucSed. " *. 

Also: Assuring ..adequate response level and validity* 

Tv/o alte^rnative sampling strategies: / . 
1. Simfjle random /sampling . 

' 2."^ Stratified random sampling-. 

_ t. ■ - ■ . . ■ , . 

C. . The, procedure most appropriate tp your s:.tuation; 
Correct responsio is . specific to- your situation. 

.. • 7. , . ■ • ■ ' 

d; The rationale for this choice: ' ' ' ■ v 

' ■ ■ \. ' ' ■ 

• Correct response is -spcbifib^ to- your choice. Be sure- you- 
have justified your choices related to items 8A and 8B above 
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List:; (pp. -57-61) ■ .. '■ " _ • .... 

ThrGG~TntTpe^4ant^ developing the instruments 

and procedures' for a desired; outcomes assessment*: ... - 



1. Write Jiigh quality items (using a number oT cf i.teriar. 

2. vyr.ite cl§ar" and useful directions. ' < 
.3.-* Plan adequately for computer use. 

The steps you will go through in this, development : • ; 

1. Generate sample items. v 

2. Pilot'^est these and revise accordingly. 

3. Select item's' to be included in the, instrument. 
■4.- D.evel^p a data colleqtiQh . in^trumenb.- . ^ 

Field test, the entire^ instrument and procedures. 




f items you have 'generated for your' instrument: 



CcirreGt response is speci 



sure your items 'mqet all tlie criteria 
and listed succinctly on p7'87 in the S'imtvlat 



iae^ individual . .However, be 
d in thGf'^module 
" — fcise. 
T 



List:. (pp. 71-79) *- : . . ' 

*■ , ■ .. * ' . ■ ^ . ' ' , ■. 
A. Three important considerations in administering thcj instru- 
ment. you have dtfeveloped: . • 

■ , ; 1. Choosing appropriate .places and times for administration 

• ' ' . ■ ■ . ■ ■ . * . p- 

• , ■ - . . . ..... 

■ 2. " Commtinicating necessary inf ormation' to all involved,. 

3. Building in*fl,exibility to; allow 'for problems . . ^ 

. . (Other qorrect responses are possrble here-^many considera- 
tions pertain) ■ . « 

'B." 'The steps you will go through, in this/ administration : 

1. " Select and train interviewers. ♦ 

2. ■ Orient respondents.; ■ ^ 

3. Develop detailed schedules for all- aspects admini.S- 
.,tration. \ * ■ * <^ " 



• 4. Provide necessiiry information* to all involved parties.- ;. 
5. Establish, an operations/headquarters;.- 
6.. Collect the datci. -V ' • " ^ 

Identif;! "three, appropriate pro.cedures involv^ed in. sumntar.izing 
and analyzing the data, you have collected: (pp..?3;-89) 

£ ■ . •. • . .■ . • 

Keypunching croraputer cards, conducting the computer jr-un, 

and interpiTetirg printout sheets. • ' \, . .. 

B. Gathering,, grouping, and integrating- added statemients and, 
open-ended responses. / . • 

C. ' Develop.ing statistiNjal 'and subgroupMveighting .procedures . . 

Summarize - three important considerations; in ^applying the- re^uli^s 
of your data analysis to a definition of the desired outcomes 
for 'ypur program (pp. ^ ^ , • • • 

«v . . ) J ■ ■ 

A. Noting items and area's^ that -rank high^ and low in the data 

Summary. ' '^'^ ' '-^ . - ' . '. ^ c2, . ' 

B. Applying the data intelligently to your situation. • ' 

G. Producing ""a document which summarizes the outcomes of the 
-.assc^ssment , and the, .process used "for it. . • 



■laerttify: ' (pp. -9 2-94) 
A. 



Three thrget ••S'^^diehces- for* yoj4r\ i?eport which summarizes 
the p'rocedures and pro^ducts of your desired 'outcomes 
assessment: . 



^. Studentfs-and the general . populace. 
2* Teachers' and administrators. 

3. Counselors and th;e Advisory group. • , ' . 

■ ■ '■ - ■ ! ' ■ " •< ' ■ : ■ ■ . • V 

.The topics of your report: • (i ' '' 

• • ■/..■■•■ 

i.- ^. Background and- Purposes of the Planning Activitic*^. 

II. Planning Activities /at All Schocjl^s ^Design , Sampling,;. 
-LnG trujnents , Data ' Collection P^'4codures , Data Analysis) 

III . • Resiilbs , CoiiclusT.ions and : Implications for Eaxph School 

IV. ' District-wide, Summary , Co-ncl-usions , and ^Recohtmendations 
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WSyS- in v;hich- the report will be tailored to each of the 
audiences listed in part A of this cjuestion: 

1.. Students and the general populace—Keep it s^ort and 
concise, avoid technical lAQguage. 

2. ' ^Teachers and admij^^ more detail.. , 

3. Gouuselors- and. the advisory group— 'Include all the 
information "necessary and relevant,, and any required 
technical langu;age • ' 





If you "want more general informatic^ on assessment, check: 




>Fit2ge<rald, Peter. Ass'essin^' the Perceived Education,al Needs* 
of Students .tf" Education, "Vol. 92, No. 3 (Feb. -Mar ., 1972) , 



13-14. 



4 .4-. . 



Kaiifman, Roger and J. Richard Harsh. Dfetermininc/ ^Educational 
Weeds—An Overview. U.S.O.-E. , 1969. ERI'C,* ED 039-631. , . 

. , ^ -•• ■ ■ : . ■ 

Popham, ^. James. Educational Needs Agsessmehy .-in .the Cognitive, 
Affective, and PsyGhomott)r Doniaini .jCehter for th.e Study of. 
. Evaluation, U.G.WA. 1969. 7pp. 

Wardrop, J. . Towards " A *^roader Concept of. Educational Assessment. 
ERIC, ED b6(ir062i , - ; • 

If you.iare interested in- learning more about testing 'in general -and 
test reliabililiY and validity, the following might be helpful: * - 

Cronbach,- L. J. Es^ehtic^ls: of psychological testing . Third^ . . 

' Edition, New York: Harper -and Row, Publishers, 1970. 

... j^, ^ . . - ' ^ J ■ " ' ■ . - . ■ ■ •• 

iloeptner, Ra.aph\ Selecting Tests- To As^'ess the Needs. Center' 
f^r the Study of Evaluation, U.b.L'.A. ■ ' " • . 

' . . ' ■ ■ ■ ■<> - ■ ■ ' . ' 

- .Isaac, S. and, Michael,: W. ■ F , Ilandbook in rese^jrch,.aj3^ evaluation ^ 



San Diego: JRober't- R. Knopp, Publisher , 4971. 



If you want to gain mqre than the el'ementary sk.ills *for calculating 
^'des^iptiye statistics * incl^udetJ in this module^ you should study, 
r^erqnces" sUch ^s these: i.- * . /, ^ 

' . E^zey, F.Pi' . A fir st reader in sta.tiBtics . Belmont, California: 
' Brooks-Cole Publ'ishing Company, .Division of Wadsworth 

Publishing Company, '1967. . * - 

■ • • ' '.^ ■ • ' ■ 

Huff', D. ancSjGeis. How to lie with statistics . New York: , 
;*Nor1?bn ♦GcAlege Divi'sidri, VV. W. Norton & Company^ - Inc. , .1954. 

Isaac, J. and V^.B. •Mi-chael. Handbook m Research and 
Evaluation . San Diego , California : . .Robert..R. =Knapp,: 

' . ..Publisher, 1971, * ■ • * 

• , ■ . ' ■ • . ' ' ' • • 

• ' Weinberx'j, G.H. and J. A. . Schumaket . S tat;i£;t ics : /yn. Intuitive . . 
Approach^ Belmont, Cal^i-fornia-: Broo.ks-Cole Publishing' 
Company;, Division of Wadsworth Publishing Company 1962 
(2nd edition) . ■ - * ; 
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If you' woultl- like more intprmation on ques tionnjiairos , try : ■ ^ 
* . * •, ' , ' . . . ' ^ .* ". . • . . ■ 

. Cramer / Stanley. The Opinion Survey as a" Rasearch .Technique. - 
^ ^ Research Guidelines For Iliglv School . Counselors , New York: " 
College Entrance Examination Board, 1966. < ' -. 

• 1968 -Negds Assessment/ Sanv Clara County Pr6-]ect to Advance ; v ♦ 
Creativity in Education ... S.an JoseV Supplementciry Education 
Center, 1968. \, ,\ ... - V ^ ' ; • 'v , . - • 

^ This is a report- from an actual assessment , - explainoJig . • 

methodology, data analysis.,, interpretation, and r-ecommehdatiaaS- 

Three examples of qUegfionnaires are: ... * 

^ ■ ■• ■ ,* 

■ . ■ ■•. ' ■ 

^ - Educational Needs As^eisment . Iowa City: . ' *■ . *. •■ 

Westinghouse Learning Corporation , 1973. 

Mgoney Problem Checklist . . New York: The - ^ : 

Psycihological Corporation, 19.50. . .'.^ 

y . . ■ • * , V ;■ - . ■ . ■^ • ■ •.. 

Priority Counseling Survey i Los Angeles : , " " 

Educators Assistance Ins titute , 1972 . 

Additional follcTw-up study information is ■ available^ in : 

m ' - ' ■ ' • ' ■ -■ ' " ' , ■ 

• .Hansen, J^ines C. and. Edwin L. Herr. :The Follow Up Study, 

Research Guidelines For High School. Counselors . New York: 
'College Entrance .Examination Board, 1966. 

^ Jacobs, James R. , Effective Follow-Up Study Procedur&s . 
The Guidance Clipic ; October, 1970, 1-3. 

if yo1a are interested in learning more about the critical incident 
technique, check the following articles: ^ * 

Flanagan, J.C. "The cx;itical incident technique." Psychological 
■ Bulletin , 1954, 51, ' 327-358.- . 

Jung, S. M. "Evaluative uses of unconventional measurement 
techniques.*'' California Educational Research Journal . 
. 1971, 22, 48-57. . - • 

If yau would like 'more information on the card-sort technique ^ 
(procedures, field tests , etc. ) contact r 

_ Youth Devclopmt^t Re'search" Program / 

American Institutes for Itescarch ^ . 

P. 0.. Box 1113 

Palo Alio, California 94302 ; 
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INSTRUCTIONS FOR INTERVIEWERS 



Introduction: Through small group interviews and a 
card-sorting procedure, we hope to have students descr'ibe 
their personal guidance^and counseling need^i^, and to 
indicate which qf those° needs they would like help to 
-resolve.- The help would come through guidance, counseling,, 
and other means of personal" assistance. The inform^ation 
wild- be completely anonymous ; students will not ne^sd to 
sign their names to anything.^ They. will be asked %o choose 
statements that describe their personal needs in four areas: 

1. Academic Learning Needs (How can I learn to learn 
. **. ^ better?). . * - . . ' 

.2. Educational and Vocational Needs (How. cah I plan ; 
better for, my presertt and future schooling and 
?■\^;ork?} * . ' ' " , ' • : * ' ^ 

3. Interpersonal Needs (liow'can I get alohg bfetter 
with 'others? ) * , * - ^ ^ ^ 

4. Intrapersonal N^eeds HIow can I imp.:^'We the way . 
^ ' I fael cibout ir^self Z) ' ' ' • . - ^ 

When all the information has been collected it. wil^L - 
be used^o assess current guidance, ^connseling aiKl ^ . ' . . • 
psycl^ological .programs and services available and v/ill ^ 
contribute to the de&igi? and evaluation of fio^ture programs , 
'and services , . The ^fpucce^s of this venture will depend i 
largely upon y^^r efforts.. •. . ^ * / • 



Por /the actual- Needs .Assessment interviews, you 'Should ^ 
hTTVe the following matei-ials with you: . . - ^ > 

. ' ."^ " . * -' 

. 1. .—Information, Forp|s . ' . ^ . ■ * " . 

2 : Docks of cards * ' " ' . .« . • ' . * • 
.3.' Additional (replacement ) blank cards for each of 
these decks \ * . * • / 

4. Pen or penoil 

You will ^i;iterv^ew individuals according to the- • 
Schedule '.v;hich you will be given. All individuals will 
lave received an prientation before the- actual assessment , 
EWch. interviewer should wojtfk with six to eight stud^^ntS; . 



GENERAL- PROCEDUIIE-. FOR NEEDS ASSESSMENT STUDENT INTERVIEWS 

* . • • • . J* ■ • 

■ ■ , ' f ' ' ' ' • ' • . • ' 

1. Stu'deut 'sciccfcs top five, needs from one 'deck.. Interviewer 
.records these.! " ^ . 

2. keep the five cards to one side.. Retain the remainder of 

^the deck . ^ \ ' ' " ' ■ 

.3, Student selects" the -top five wants; reariranging: the remainsjcrr 
: of the deck ifVthe wants are not the same as .tlio . needs** / 
Interviewer records thc'se. Interviewee put? top five, wants 
to one' side and reassembles rest of deck. ^ . 

4. ^. Repeat steps ond through, three with each o^f the remaining 

three decks. 

5. .From the new " deck" of 20 • (or fey;er) wants , select the:top 

eight wants. Interviewer records these. Interviewee 
replaces all cards in proper deck. ' . , 

5.' Fill in open-e-nded question .#4 ^(second page of- interview 

.sheet) if \jiev/. wants have been added on blank cards that eijd 
up in top 20 wants. . ; - e 

7.., Have interviewee fill in r^st of open ended questions on 

. page 2, then return form to interviewer.' * — 

8. ; Remove and destroy blank cards on which additional needs 

were written. Replace destroyed blank ca^^d^with new "blank 
cards, that hctve beeri prpperly numbered on baqk . - 

9. Shuffle cards in each deck and put head&r bard on top of . ^ 
each deck. - 

- ■ . \ , " ' * " I - ■ ■ ■ 

The follov/ing step- by step instructions will help you in 
conducting the int.erviev;. ' : 

A. . Introduce yourself. -Explain the purpose of the assessment, 
• ahd the face that students h.avG .a'real opportunity to have 

an impact on programs that affect them. Emphasize ' the • 
importance c5f their being honest and saying v/hat they feel. . 
Hand each student in your group an information form. Read 
' theC statement on the front page as they folJLow along. 

B. Have the ;5tudqnt9s, f ill in the top of the data' collection 
form. Explain -tba^ -the assessment is anonymous and no one 
will know what, they Sc^. 'Walk around ^and doublje check their 
responsos .tc> make sure they are axiswering properly.' Collect 

^, the information Forms.- •Arrange these forms in front of you 
to corres'pon'd to the way in. which the group members ixrc. 
S43ated.. Give Qach person in your group a number, and then 
lightly write tKat number on theit form so that you will be • 
•sure not to cjet the -forms mixed up. . Give each student a dock 
of ceyrds^ -The 'students- in you-r groups- s):ould start on differ- 
ent colored decks. Give the first student the Green deck. 
-To the second, give the -Blue; the^ third, the. Orange; and ^ 

the fourths the Yellov;. Continue' this cycle for the other^. . 
• ^membdirs of your group,' always procdeding in the Gr©en-Blue- 
Orange-yellow order. . « • 

— ■ • ■ ' • ' ■ ' • — ■ • — — 
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C. ' Explain the. nature; of the four decks of cards. Explain to 
students the procedure for sorting tbe cards in the deck 
* they are working On into statements which do represent Heeds 
for them/ and those^, which, don' t. Explain rlfat if they have 
a question they should ask you. Explain that if they have ' 
a need in the area dealt with that is not already included^ 
they m.py vfltite it (them) on the /blank card (s) at the back of 

the deek4 ^ ' ' » - c f 

"iSfhdn a question is asked, hesitate to put your ideas ^ 

in the student '..9 mind. First , try to g^^t his understanding 
, of the card's meaning. Then, respond^ to the' accuracy of 
that statement. . ' • 

When»a student finishes with* a deck, h^nd him the next . 
deck in the Green-Blue-O.rar^ge-Yellow codor sequence . Also, 
remember what, color deck the last student in your group 
started with. Continue ^the Green-Biue-Orange-Xollow cycle 
for all students' you work with during the week. This is ^ 
done to counterbalance the admini,stration procedures. 
Because some students work more 'diligently on the firs^ deck 
they receive than the last, using this procedure will enable 
each deck to be presented first an equal number of times. 

-D?^, Expl ciiii Li re 'pro^ess'^^f selecting the five most important 
L ■ cards from the' pile that represents chosen needs; Respond- 
ents may select less than five, but not more. "Then explain 
the process of putting the most important n^ed of the five 
face up, follov/ed by. the second most important, and so on. 
When the pile is complete, the respondent' turns the stack 
of five cards over and hands them to you so that only. the 
numbers on the back of the cards -^re showing. 

Wait for the students to select their five greatfest 
needs. Wjien they have done this', you record the numbers on 
the back of the cards in the appropriate spaces under "NEEDS" 
on the data sheet. The number 'On the top card will be the . 
number of his first choice need, etc. Make^ure the inter- 
viewees know "you are looking ^t the card numbers only, not 
at the statements. 'This should add to their feeling that 
the results are treated confidentially. 

If ,a student has.wr-itten a need on= a blank' card as • onei 
of his five- top needs,, record the number in the data section. 
Do not read tlie need which the student has written. Later, 
after the interview has envied you will have the student trans- 
fer the need statement*^ from the card to the appropriate sec- 
tion of Question #4 on the second page of the data sheet. -If 
a student wrpte. a need on a card but did not select it as one 
of his five greatest needs, have him keep that card aside 
until the end of the^ interviev/ and then destroy 'it. When a 
student writes on a blank dard,- be sure to prepare a replace- 
men,t card 'and insert it in the ideck before the "d:Qpk .is. reused. 

Do not allow .the. students to record their choices on 
their own' data sheet. Some students may want to do th^s , but 
in the interest of. accuracy, you should do air the recording. 
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E. ' Colldct information about respondent? V wants, ^sk them to 

imagine that the best possibl^ help is available. . Ask them 
to select from the card statements.th6.se with which they* 
would like help. These "wants" choices may be different. \ 
from the "need" choices. Have respondents select their top 
fAVe (or les"s) want?* using the process outlined : in step D 
for each deck- T;his timev h<)wev;^r , have respondents keep 
their top five "want?"- cards from each deck after the 
numbers are recorded. - 

F. With needs and wants recorded for each of the four areas, 
have respondents combine their top f ive wants cards, into . . 
one deck of 20 cards , and select the top eight wants state- 
ments from th6se. This gives top wants across all cate-^ . 
gories. Follow the same Recording procedure, and return 
the .cards to the respbndents. ^ ^ 

G. Have the 'respondents reizurn their wants cards to tlie appro- 
priate decksT^and mix them through the decks,. Have them ^ 
remove cards on which they wrote additional statements , theh 
shuffle the decks, thoroughly. Place the header card at the 
front 'and blarik^cards at the end. Collect the reassembled' 

. ' . decks and return the. data sheet to each respondent. 

H. Have the respondents . copy the statements they wrote on blanH 
cards in the space under #4. Then have . them pifocecd v/ith 
questions five through eight oti the data collection instru-- 
mant. Emphasize again the importance of honesty, £(hd the 
•need for suggestions and constructive criticism. . Have - 
respondents destroy their added written statements (cards) 
after they hgive been recorded on tlTelJ^^ 

ments. ' When everyone is through, collect these forms." 
Check respondents, to be sure they are filling but t\v^ forms 
correctly. . 

I. Explain again^brief ly how the information will be used, and 
how it should benefit respondents. Thank them for their 
cooperation. Have stufpnts return to their regular schedule 
in the way least disrujptlve to the school . (this should be 
predetermined) . 

AFTER. THE INTERVIEW., . . ' ^ : / 

" ■ .• . ' ■ ' . r** ' ' 

J. Assemble all the informatic^n forms and the decks and return 
them' to assessment headquarters. Keep them in a separate 
file or envelope until you turn them in. 

... ' . . * • 

K. Replace any used blank cards at the back of the dQjpk/ num- 
bering them correctly (see item D above). 



STUDENT ORIENTATION. 



To the. Student . i ' . 

Your high school is engagitig in a project to review - 
and •improve the guidance and counseling assistance- you 
receive in ^schooi*— -Withi the next few days you are going 
to have an opportunity to say what your guidance and coun^ 
seling needs are. The . inTformatiori you give will be anony- 
mous ; you will not have to. Sign your name to anything.- It 
is not possible.. to collect information from ajl ^students, 
3o a raindom sample of ninth through twelfth grade students 
has been selected.^ Your name may. be in that saaiple. lit 
addition to information from, students^"- a random sample of 
\scho6l. personnel and parents will be. used to make sure that 
gtiidance and counseling programs in this school are directed 
toward the most importan't needs that you 4. the students / haye 

During the next v;eekv think about the types pf personal 
^nd educ^itional 'things with which you nieed help and don't 
worry about hov; hard it might be to get. that help; Imagine 
that ^very possible way of meeting your needs^, could be made 
available. Try to be completely hdnest; don' t . report what . 
you think others would like to heair. This is a real oppor- 
. tunityfor you to have aniimpact on what happens to you in 
school. 

You will bcL asked to__s]JUDpJ.y_jbh^^ 
heeds by using a ca;:d-sorting procedure. You will b,e given 
four decks of cards. 6A each card a different nee^3 will be 

•written. If the cards which you are given do not describe* . 

* all of your needs^ blank^ cards, will be available on. which 
you may write other needs you may have. You also will -have 
a chancy to make any general commentsj you feel are important 

You Will work i'n small groups with an^assistant li^ke^, 
the per'son who is handing out this sheet today. Since this 
information, on your needs can have a real ipipact on the 
guidance and counseling which is. available to you, your 
' thoughtful cooperation is needed. 



Your Guidance and Counseling , Department 



WHY NEEDS ASSESSMENT? 



V Why should you , as a student /' be worried novNabout the 
job situation a few y«2ars from how, ^pr -about plahnihg for 
your future .and making some determinations of your own^ needs 
fo^ that future? Consider briefly the following factsV" 



/ . 90% of all the scientists who ever lived are: aliV-e—r- 
today . 

' Human knowledge is doubling about every, .15 years. ' 

. There is more change in the lifespan of one man ' 
today than there was in the spacp of 1 , 000 years^ 
a few centuries ago. - 

. The darth's urban 'population is doubling about 

every eleven years. . There are more than twice as ■ ^ 

many people living in. the cities- of the world 
tod^iy as when you were born. 

' 4 ' Ha3r£ of al^ this energy c^^^ in the past^ 

2/000 years has been y6exnstnned in the. last 100. 

. By the time today ' s/ teenager reaches old age, the 
society around him will be producing rpughly 32 
times as„ much in the way of goods and services as 

- - ■■- v/heii he was born. - . • - . ■ - ;•• ■ " ' ^ ; r"- 

' ' ' - . ' ' . .' . ^ ' ■ ' 

Facts taken fro% Future Sphock 
by Alvin Toffler , _ 

We live in a world unlik^e any that has ever existed _j. . v ' . . 
previously. The, pace of change haB reacFied such lightning 
proportions that it is only through thoughtful assessment 
of one.'. < pv/n place in the world and intelligent planning 
for the future that one can adapt and . survive. . 

^*The 'classical pattern of an individual having a schooling 
phase, a working, phase, and a retirement phase is changing fast. 
A student leaving ' school is finding that he must resume his - 
studies again and again if he desires to keep up with, the inci^case 
im. scientific and technical knowledge. He must aot only renew 
his knowij-edge of his own. special area; he must bq ready* to 
prepare either^ fo^ a nev; occupation within vthe same occupational' 
arqa, or even for an entirely different occupational field. . * ~ 
Since an indivi dual' s lifetime will be filled with decision 
points' -more than ever. befareV teaching students to adapt to 
change is rapidly becoming a primary- aim of American education." 



Dr. Jack A. Hamilton, Educator 
and Research Scientist 



So today's student is presented with an unprecedented 
challenge by the world he is about "to enter. Every effort he 
can make to respond to this challenge, to come to knov/ himself 
and the vrorld in which he will live more full^f may represent, 
the difference Ipetweeri survival and being left behind and lost 
by a world hurtling at breakneck pace into the future. The 
interviews and assessment atoout to be conducted "represent one 
response: an attempt to determine what you as human. beings 
need now, and this will iead^to an attempt' to then respond to 
these needs in such a way that,, hopefully , you wj.ll be able to 
deal effectively with the world of tomorrow. 
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Tape-Slide introduction 
DESIRED •OUTCO^Ui:S •ASSESSMENT' 



VIS.yAIi 



AUDIO 



1. LONE YOUTH LOOKING 7\T 
A NOTICE WHICH READS:, 

. BOARD MEETING TONIGHT, 
' OPEN TO PARENTS AND 
ADULTS. • 

2 . SAflE YOUTH WATCHING 
^T.V., PUBLIC SERVICE 

. ANNOUNCEMENT. 



3. SAME YOUTH GLANCING 
IN ROOM WHERE A. " 

' ' TEACiiERS ' MEETING , 
. IS GOING ON. 

4. SAME YOUTH VJALKING 
^ llOME , , ALONE , HEAD" 

DOWN, AND DEPRESSED. 



5. TITLE: DESIRED. OUT- 
V COMES ASSESSMENT. ' 

^ -STAFF DEVELOPMENT 
_ PACKAGE . * . 

fr. COUNSELOR .IN HIS ^ 

office with a student 
'seeking help. - 



CARTOON. FIGURE 
STUDYING EXPRESSION 
SERiibuS, IDEAS COMING 
OUT ALL SIDES OF HIS 
HEAD. 



1. Background hall noise during 
change of class * 



T.V. Announcer's Voice: " . * .a^^ 
so if you ' re over is , be sure 
to vo\g in next Tuesday's 
election . " , : 

Meeting noise. Someone talking 
about what studentJs need. 



Youth -saying to self: "I wish 
someone v/ould ask me v;ha.t I 
want sometime. Don' t my' opinions 
count?" 



7. 



M^ale NaTrator: The main idea - " 
behind a desired 'outcomes assess7 
ment is' finding out from students 
what they* warit . ^ 

* If done veir/"this is a. sizeable 
task. It is also 'a' rewarding 
one, 'a*nd one that can .pra4^ce 
significant lasting benefits . 



The module for which. this 
slide- tape. presentation serves 
as an introduction is designed 
to\ provide you tvith the know- 
ledge and skills .nece„aaaLr¥-JtO-^ 
conduct such a-s'sessmcnt . ' - 




DIl^GRAT^ OF THE MODEL, 
AS - I^APPilARS ON 
PAGE 2 THE MODULE 



CARTOON. FIGURE LOOK- 9. 

ING AT AN IMPRESSIVE 

LIST OF SKILLS ON THE i 

BOARD, POINTING TO SELF, . 

WITH EXPRESSION WHICH 

SAYS: "mib, -ME?" 

CARTOON. - FIGURE MADLY 10. 
WORKING AWAY, PAIi'ERS 

FLYING IN All directions 



CARTOON. FIGURE WITH 
SMUG look', sitting 
WITH GTOUP OF LIKE . 
CHARACTERS. 

CARTOON. FIGURE LOOKING 
AT PAPERS LABELED: 

APPLICATION PROCEDUl^^ST^ 
POST-TEwST. • ^ • . ' 



CARTOON.. .FIGURE WITH. 
CONFUSED EXPRESSION, ' 
AND "CU RRISnT status * 
iYSHEBSHENT , " .'''DESIRED 
OUTCOMES ASSESSMENT 
FLOATING AROUND HIS HEAD. 



DIAGRAM: 
■ A ' NEED 



.1 



B 



WHERE YOU 
ARE NOW 



WHERE YOU 
WOULD- LIKE 
TO BE 



A LIST OF PRIORITIES., 
NUMBERED, INTENDED. ' 
TO' REPRESENT PRESENT 
PRIORITIES. 



The module begins by showing 
you the irelationship between 
this module and the overall j; 
model for program development 
of which it is a part. . As you 
can see, this module comes in 
the initial rplan^ing sta^e, • 
providing information uadful 
for s.ubsequent steps in the. 
model. 

Then the module lists all/ the 
knowledge and skills you ; 
should achieve by working 
through it. , . • 



As you work through the module > 
you will .encounter frequent . 
progress checks to show you ^ 
wnetner you are "absorbing - 



what you're, reading. 

11. Group activities are also in- 
eluded to • let you share your 
knowledge with others, and 

benefit from theirs;. ^ , 

i 

12. Exercises at the end of i^he 

. module ;v/ill let you prabtice 
. your skills, apply them^to 
— — d make ' 

sure you have learne3^'*eWryt'lTrTrg^ 
you* re su-pjDtDsed to. 

13. Female Narrator: ' Perhaps you're 
wondering exactly what a desired 
outcomes assessment is. 



14.*^ Need can br? defined as the gap 
between whQ,rc yo,u are now, 
Pdin^ A* on the diagram, and 
v;h'ere you v;ould like to be , 
.Point B. 



15.. The current status assessment, 
pxpladned in the previous 
module of. this series, deals^ 
with Point Ar ' determining where' 
you are now. 



S'TUDENTS IN A .DISCUS- 
SION GROUP, LED BY A. 
TEACHER.. 



CARTOON. '^FIGURE WITH 
. RULER, MEASURING ON A 
DESK. 



CAIOTOON: figure TRYING 18, 

'to decide ' wi-ii.cii OF 
several paths to take ^ 
through a woods, : 



A PAGE FROM A 
QUESTIONNAIRE, 



A MAN DROPPING 'many 
"LETTERS IN. A MAILBOX. 



16. ^This module deals with Point 

where you would like to toe. ^ ' 
This is .determined through. ' - 
asking st'udents, and possibly, 
some adult subgroups as well^ * 
v;hat students* needs are and 
what they would like help with^ 
if the besti possible services 

• were available. 

17. Fi/ally, the. gap between what 'j. 
the desired outcomes assessment 
shows and what the current 
status assessment shows deterr 
mines in what directions and 

- - how^^f ar -your program should move. 
How to precisely determine this 
gap and wlrite program go als a nd 
objectives ds dealt with in the 
next module. . 

A nun^ber of approaches have 
been used in the/past to . 
> measure stu<^ent needs -.and deter- 
mine program desired outcomes., 

19. The most common is the question- 
naire, a paper and pencil 
approach' which has students * i 

* check, rank, rate, or v/rite- 
responses toitems. 

20. ^ Follow-up studies sample sub- 
. jects who have loft a program 

and know from. their subsequent 
exp/!rience what the Ip^hey would* 
like to have had. 



A, STUDENT SORT'lNG 

A DECK OF NEEDS CARDS. 



CHART . SKVEN STEPS . 
IN THE: ASSESSMENT PROCESS 
(from, page 23 in the | , 
module) * . ^ . . 



21. This module concentrates on the 
card sort approach, cJhe whicl? 

, actively invQlves students and ^ 
adults in the ""assessment . ^' 

process apd provides comprehen- 
sive and usefully structured 
data. * V 

* ' ■ • ■ 

22. * Whatever the appraach, " the assess 

ment process has certain well 
defined steps. The seven shovm 
on the chart are the m^jor ones* 
(Allow time for reading these. ) ' 



1'.. 



''' ' iDE^I'^^^^^^^ 



24. 



25. 



26^. 



27. 



BLACKBOARD WlTH ^nr • 

.y-^P^HS^o'^AL- 
Snf ^ ■ ?r^^^^SONAL ■ 

vucATIONAL NEEDS? 
AS ABOVE WITH I-rnpnc ' 
-.AGE? -SCHOOL? iTEArirPDo^ 

■ ™?^f:„»CNSEtoES- WORK- ■ 

- ' i 

A GROUP. OF STUDEKLTQ ' 
B^ING .QRiENa-ED™ 



28. 

2°<3f' 



;SHOT OF A COMPUTER. 
WVe;,^S„£ET pop a . 



*23. 



24. 



• ■ * • ■ "7 

Male NaK-ator: #• 

gr-oup, layg the arn.?^^^^ advisor 

^Next, the preciQA^ 



\ 



26, 



27, 



28. 
29. 



° - . GARi'OON. FIGURES 

;3^^SuS^r-- 

'LFFORT. 



30. 



31; 



The instruments must°i.o " • " 
Pilot tested, revKL^ written, 

The Sixth-Sten "in.r,,l- 

together ancKanaJ^? ""^^ Pulling, 

' ' analyzing the data.. ' 

the result? anH ? summarize . 
as th.y Have been^de'^e^S"-^ . 

Female Narrator- An^ " .'. 

^one, exac?i;^,i'^?^ '^^^^ 
accomplished? Mston '''' 
answers to this^'u^'^tio". 

"This has given h^' t-u , . 
education." "^"^^^l^tj^-into 
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32. 



33. 



34, 



35; 



3 6. 



37. 



38 



PROJECT DIRECTOR, 
liEADING PLANNING • \ 
SESSION- (BLAGKBOARD '■ 
INFORMATION PROVIDING 
IDENTIFICATION) -. 

YOUTH FROM FIRST FOUR 
•SLIDES, TALKING TO 
FRIEND. 



COUNSELOR ^ COMMENTING 
IN PLANNING SESSION. 



ANOTHER COUNSELOR, 
SAME ■ ' 



GROUP OF STUDENTS IN 
GROUP COUNSELING SES- 
SION, WITIl ONE OF ABOVE 
CpUNSELORS. 

INDIVIDUAL COUNSELOR 
WORKING BY HIMSELF.- 




FAIRLY LONG RANGE SHOT 
OF THE SCHOOL BUILDING. 



39. YOUTH FROM FIRST FOUR 
' SLIDES, WALKING HOME", 

CHEERFUL EXPRESSION , 
.. 'HEAD UP. ■ - 



40. COVER PAGE FROM THE 
-. MdOULE. , ■ . . 



32 . .. '"jWG„ have concrete, eitipi-rical 
data to work from now. We 
don't have to plan from vague 

' .6 theories or guesses." 



33. "They did it. They asked me 

. .what I^ needed. I f eci like - 
I've had a little say in some^ 
thiog for a change." , . 

3 4> "I find we aire, 'working together. 
' better as a team,- now thatf "we 
have common and reliable in- 
formation oh .which' to* base our 
efforts. " ■ ■ / . 

"We have begun a process of 
planning and evalijiatipn, whi.ch 
should keep us from stagnating, 
and help us ^tay abreast of 
Te changes and needs',". 

36. Female Narrator: . As these com- 
ments suggest, there are many 
advantages to conducting a de- 
sired outcomes .assessment* 

37. The mpduLe will, require some 
harjd w.(pr\k. It's designed to 
take £rom 10 to 15 hours ,to 
complCite. 'It may become 
frustrating at. times. 

38. But/ it can help to give yoii 
im^ortan/t sk-il'ls useful for 
marking real improveme'nts' in - 
your guidance, counseling ; and 

/iacement program. 

39. /it is one t^ooli for, making edu- 
/ cation responsive to young 
/ people, and structuring pro- 
^ grams.: aground their needs. - 



40. 



Male Narrator: At, this point 
you may begin work oh the ' . 
module. 
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I. 



II 



PROCEDURAL CHECKLIST 

Assessment Process 

• . ■ • > " ' - ■ 

Plan and coordinate with the advisory giroups . 
A, 



B. 



Obtain .approval and support from appro- 
priate groups.. • . ( . , 

.Acceptance and participation of various 
■ sub-gro.ups . ' 

Other .qonsiderations': 

1/ Who will select areas? 
♦ - 

Who will define sample? 



2. 
3. 

4. 



Who v/ill choose or develop instru- 
ments? » 

Who will be in charge of 'admihis- 
trat ion7 . 



How much help? 

Wfio will'.- analyze the data? 



6>" Who need§^ to be, involved at each 
step -arid hbw will communication 

Qccur? , 

» ... 

7. vJho will coordinate with computer 
so data will be usable? 

' 8. Who. will maintain^ a : record of phe 
program developmdfnt? 

Identify areas on which the assessment 
will f pcLis . \ ■' ' \ ^ . 

\. . ' . ■ . ■ "■ t ^i-^ ■ ■ ' - 

A. ; What is the central purpose?, Compre- 

hens^ive;, or 1 or 2 key areas? 

B. Is assessment for alT or only pertain 
sub-^groups? 

. •' ' >■ ■ .' ■■ • 136-.' 
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II. Defining and -selecting the sample (10-15% 
of population) . . ^ 

^ A. The few who are measured must reflect 
. the will„ of all. . 

• B-^ "Random sample--QnlY' chance determines 
who participates* 

C. stratified sampl'e — balance sample with 
key factors . 

.p. What level of response will be considered 
• ■ ■ adequate?' ' ' . 

' •■. ■ ^ 

IV. Developing the instruments. 

■ A. What is evaluation of available cards 
and statements? 

. ' 1. Does revision need to be assigned to.. 
' ' someone? " 

2. What arfeas are to be Included in ■ ■ 
' • ' survey? . ^ 

3. What statements need to be added?-- 

B> - Can we pilot test these' on an adult or 
student i^roup?" * ■ i 

C. Who will develop a 'data collection 
instrument? 

'' "1. liow v/ill responses be recorded? 

2. Hov; will responses , be summarized? 

■* D. ilow is data to be reported? ' 

1. ' Number of times Chosen. / 

' ■ ■ ■ . \ ■ 

2. Which was chosen most i^mportant most 

often? . . 

* ' * ' \ . 

* 3. - Weighted mean. 

■ ' 4 . Ilpw; should data be grouped? Subpop- 
uiations, sqx, grade, age? . : ^ 
' . . » ■ . / , 
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Administering- the instrument 

A. Who will do it? Training? * Counselors^ 
teachers, aides , substitutes , graduate 
students., residents? . ■ ■ . ' 



B . 
C. 

D. 



Schedule: where, when? 

Are written directions needed? For 
st^udents and interviewers? 



Preparing students. 
. 1 . ■ Purpose . 
2. What they, can gaih. 
-3. What is expected. 
Summarizing; and analyzing data. 
A. 



B. 

C. 

D.. 



Determine best form-and method with 
computer' people.- 

How are open-ended questions to be ^ 
handled? - ^ . 

Who is going to do the .analysis? 

IIpw will data be weighted for -Analysis? 

1. First choices. 

Relative .percentages , 

3 : _ ■ ' ...... ; ■ ■■ 

How w^ll "figures be 'reconciled? 
How will v;eighted 'mean be utilized? 



2- 
3. 
4. 
5. 



Hqw will responses of various sub- 
groups ho weighted? 



Applying the results^ to -a definition of the 
desired outcomes.* 

A. Consider both highs and' lows ir* decision 
. making process^ ; 

B. ^./ Is data., to be considered as a w^ole or 

■ divided into subgroups (i.e., grade,!, age, 
sex,,* etc. j ? 

C^ Who will produce document reviewing 
• I ' process and results? 



' 1 

■ '■ 
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: •" T STAFF' TRAINING MODULE DESCRIPTIONS 

1. drieritcftion . 



2. 



3. 



in-depth approach to 'understanding what the compre- 
a^pproaC'h to guidance, counseling, and placjement is # 



Gives an 

hensive a^pproc _ ... 

• -'What its advantages. ^^^^ competency-based training xs 

related to using thisT'^approach, . and how to deterinihe what 
training is relevant. Requires 15 - 20>hdurs to complete. 




Assessing Desired Oufacomes . - ^ 

Presents various methods of planning and conducting a needs 
assessmditit as well as ways of summarizing/ ana'^iyzing, and 
reporting the datal Requires 1^ - 20 hours to complete. 

Computer .Assisted Reporting 

Provides a general^ knowledge of tlie prpcess necessary to 
develop an Effective Computer Information System and enables 
the participant to select the best approach for gathering and 
analyzing data from all available options, for a district's 
particular- needs Requires 10 - 12^hours to complete. 

Cu rrent Guidance Program Assessment ' ^ 

— '■ — '• ~ ' ' '' ' 

Provides an understanding of why .a current guidance program 
assessmiJnt is needed, how the assessment can be conducp.tec3^, 
and what use can be made of the data after it is obtained. 
Requires 10 ^ 12 hours to complete. ' 

5 . Determining Program Goafs and Objectives . . -> . 

Makes use of the current status assessment and desired, out-* 
comes assessment reports to produce a complete set of program 
goals and studqnt performance . objectives for a new program. 
R^quires^ 10 - 12\rio,]Lirs to complete. v ^ 

6. Alt ernabivG Strategics , 

Examines strategies which would be. appropriate to the ^goals 
' and objectives of the program being structured, specifies ; 
criteria for assessment of alternative strategies presented, 
and provides evaluation procedures for the selected strategies, 
Requires 12 - 15 hours to complete. ' . . 
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Presents an overview / • 

hensive daJaerau?^^ counseling groSpl Jo^L","'" '^s* groups, 

useful in facili'tftl„r?^^^^^° conmunication ^Jfif^^^^'^ts* 
-"tings. Reg-uJIes i"I i-^'f P«s6nal interlotion ' 
pr, „ . "ours to complete/ 9"up 

^^^"'^'^^^^SraErofessio^^ 



Provides the structure. ^ • 

, ^2 15 hours to complete "^'""^ °^ P^^Prpf^ssionals 



